This document describes how to use CRD to configure the log collection feature of TKE Serverless cluster.
Log in to the TKE console, and enable the log collection feature for the serverless cluster. For more information, see Enabling Log Collection.
You can take the following actions to configure after enabling the log collection feature for the cluster:
After enabling the log collection, you need to configure the log rules including the log source, consumer end, log parsing method, and so on.
Select Container standard output as the collection type, and configure the log source as needed, as shown below:
This type of log source supports:
NoteFor container standard output and container files, besides the original log content, the metadata related to the container or Kubernetes (such as the name of the Pod that generated the logs) will also be reported to the CLS. Therefore, when viewing logs, users can trace the log source or search based on the container identifier or characteristics (such as container name and labels).
The metadata related to the container or Kubernetes is shown in the table below:
Field Name Description cluster_id The ID of the cluster to which logs belong container_name The name of the container to which logs belong image_name The image name IP of the container to which logs belong namespace The namespace of the Pod to which logs belong pod_uid The UID of the Pod to which logs belong pod_name The name of the Pod to which logs belong pod_ip The IP of the Pod to which logs belong pod_lable_{label name} The labels of the Pod to which logs belong (for example, if a Pod has two labels: app=nginx and env=prod, the reported log will have two metadata entries attached: pod_label_app:nginx and pod_label_env:prod).
Note
- CLS currently only supports log collection and reporting for intra-region container clusters.
- The log set and log topic cannot be updated after the log rule is configured.
Parsing Mode | Description | Related Document |
---|---|---|
Full text in a single line | A log contains only one line of content, and the line break `\n` to mark the end of a log. Each log will be parsed into a complete string with CONTENT as the key value. When log Index is enabled, you can search for log content via full-text search. The time attribute of a log is determined by the collection time. | Full Text in a Single Line |
Full text in multi lines | A log with full text in multi lines spans multiple lines and a first-line regular expression is used for match. When a log in a line matches the preset regular expression, it is considered as the beginning of a log, and the next matching line will be the end mark of the log. A default key value, CONTENT, will be set as well. The time attribute of a log is determined by the collection time. The regular expression can be generated automatically. | Full Text in Multi Lines |
Single line - full regex | The single-line - full regular expression mode is a log parsing mode where multiple key-value pairs can be extracted from a complete log. When configuring the single-line - full regular expression mode, you need to enter a sample log first and then customize your regular expression. After the configuration is completed, the system will extract the corresponding key-value pairs according to the capture group in the regular expression. The regular expression can be generated automatically. | Full Regular Format (Single-Line) |
Multiple lines - full regex | The multi-line - full regular expression mode is a log parsing mode where multiple key-value pairs can be extracted from a complete piece of log data that spans multiple lines in a log text file (such as Java program logs) based on a regular expression. When configuring the multi-line - full regular expression mode, you need to enter a sample log first and then customize your regular expression. After the configuration is completed, the system will extract the corresponding key-value pairs according to the capture group in the regular expression. The regular expression can be generated automatically. | Full Regular Format (Multi-Line) |
JSON | A JSON log automatically extracts the key at the first layer as the field name and the value at the first layer as the field value to implement structured processing of the entire log. Each complete log ends with a line break `\n`. | JSON Format |
Separator | In a separator log, the entire log data can be structured according to the specified separator, and each complete log ends with a line break `\n`. When CLS processes separator logs, you need to define a unique key for each separate field. Invalid fields, which are fields that need not be collected, can be left blank. However, you cannot leave all fields blank. | Separator Format |
When you select Multiple lines - full regex, Single line - full regex, or Multi-line texts, the regular expression can automatically generated based on the log sample.
Here takes the Single line - full regex as an example:
NoteCurrently, one log topic supports only one collection configuration. Ensure that all container logs that adopt the log topic can accept the log parsing method that you choose. If you create different collection configurations under the same log topic, the earlier collection configurations will be overwritten.
Was this page helpful?