- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Development Guide
- Design of Data Table
- Importing Data
- Import Overview
- Import Methods
- Synchronizing Data with JDBC
- Synchronizing Data Through External Tables
- Real-time or Batch Writing of MySQL Data
- Importing Data from Kafka
- Importing with DataX
- Importing Doris from Logstash
- Data Transformation, Column Mapping and Filtering
- Importing Strict Mode
- Importing Data in JSON Format
- Exporting Data
- Basic Feature
- Query Optimization
- Ecological Expansion Feature
- Product Policy
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Development Guide
- Design of Data Table
- Importing Data
- Import Overview
- Import Methods
- Synchronizing Data with JDBC
- Synchronizing Data Through External Tables
- Real-time or Batch Writing of MySQL Data
- Importing Data from Kafka
- Importing with DataX
- Importing Doris from Logstash
- Data Transformation, Column Mapping and Filtering
- Importing Strict Mode
- Importing Data in JSON Format
- Exporting Data
- Basic Feature
- Query Optimization
- Ecological Expansion Feature
- Product Policy
Tencent Cloud TCHouse-D supports log collection, and you can enable Log Service for clusters as needed. Log Service supports both ES and CLS forms.
Note
ES service currently only supports the Beijing, Shanghai, Guangzhou. For billing information, see ES Billing Overview.
CLS is a type of postpaid service. Please keep sufficient funds to avoid affecting your log upload and display. For billing information, see Log Service Billing Overview.
Enabling Log Service
You can enable Log Service when creating a new cluster or at any time while the cluster is running. Such operations are explained in detail below.
Enabling During New Cluster Creation
During the creation of a cluster, you can choose to enable Log Service as needed. If you choose ES Log Service (currently supports Beijing, Shanghai, Guangzhou regions), you need to select a project space for management. If you have never created it before, click Create new project space to quickly create it. The log analysis feature is provided by ES Serverless service. For details, see ES Serverless Service Overview.
If you choose CLS Log Service, you need to select or create a CLS log set in the same region as the cluster. CLS logs are stored for 30 days by default.
Note
A new log Topic will be created in the log set you configured. You can preview your log Topic on the CLS Log Service page. Do not delete the configured log topic; otherwise it will cause the log retrieval page to query failure.
Enabling or Modifying Existing Clusters
If you didn't enable Log Service when creating the cluster, you can log in to the Tencent Cloud TCHouse-D Console at any time and configure the Log Service through Operation > More > Create log service configuration in the cluster list. For unauthorized users, you need to authorize first, and then configure the log set.
Log Search
After enabling Log Service, you can easily retrieve logs from the console to assist in operations.
Operation Instruction
1. Log in to Tencent Cloud TCHouse-D Console, click Cluster ID/Name from the Cluster List to enter cluster details page, and switch to the Log Analysis page.
2. Both Node Log and Search page modes are supported. In the Node Log mode, you can view logs by node IP and log type, and perform log analysis using features such as specifying time ranges and keyword search.
3. In Search mode, you can conveniently set search keywords, retrieve logs within a selected time range, and group the filtered results by node IP. The syntax and rules of keywords are detailed in Syntax and Rules. The query results display the most recent 100 records for each node by default.
How to Correctly Read FE/BE Log?
In many cases, problems need to be investigated through logs, so here we explain the format and view methods of FE/BE logs.
FE Log
FE log mainly includes:
fe.log: main log. It includes all content except fe.out.
Fe.warn.log: subset of the main log. It records only WARN and ERROR level logs.
A typical example of an FE log is as follows:
2021-09-16 23:13:22,502 INFO (tablet scheduler|43) [BeLoadRebalancer.selectAlternativeTabletsForCluster():85] cluster is balance: default_cluster with medium: HDD. skip
Where:
2021-09-16 23:13:22,502: log time.INFO: log level, INFO by default.(tablet scheduler|43): thread name and thread id. Through the thread id, we can view the context information of this thread and facilitate troubleshooting.BeLoadRebalancer.selectAlternativeTabletsForCluster():85: class name, method name, and line number.cluster is balance xxx: log content.BE Log
BE log mainly includes:
be.INFO: main log. This is actually a soft link, linked to the latest be.INFO.xxxx.
be.WARNING: subset of the main log. It only records WARN and FATAL level logs. This is a soft link, linked to the newest be.WARN.xxxx.
A typical example of an BE log is as follows:
I0916 23:21:22.038795 28087 task_worker_pool.cpp:1594] finish report TASK. master host: 10.10.10.10, port: 9222
I0916 23:21:22.038795: log level and date and time. Capital letter I indicates INFO, W indicates WARN, and F indicates FATAL.28087: Thread id. With the thread id, you can view the context information of the thread, making it easier to troubleshoot the thread.task_worker_pool.cpp:1594: code file and line number.finish report TASK xxx: log content.
Yes
No
Was this page helpful?