- Release Notes and Announcements
- User Guide
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Resource Management
- Permission Management
- Log Collection
- Collection Overview
- Collecting Logs in Self-Built Kubernetes Cluster
- Collecting Syslog
- Collection by LogListener
- Collecting Text Log
- Uploading Log over Kafka
- Uploading Logs via Anonymous Write
- Uploading Logs via Logback Appender
- Uploading Logs via Log4j Appender
- Uploading Log via SDK
- Uploading Log via API
- Importing Data
- Tencent Cloud Service Log Access
- Metric Collection
- Log Storage
- Metric Storage
- Search and Analysis (Log Topic)
- Overview of Search and Analysis
- Syntax and Rules
- Statistical Analysis (SQL)
- Quick Analysis
- SQL Syntax
- SQL Functions
- String Function
- Date and Time Functions
- IP Geographic Function
- URL Function
- Mathematical Calculation Functions
- Mathematical Statistical Function
- General Aggregate Function
- Geospatial Function
- Binary String Function
- Estimation Function
- Type Conversion Function
- Logical Function
- Operators
- Bitwise Operation
- Regular Expression Function
- Lambda Function
- Conditional Expressions
- Array Functions
- Interval-Valued Comparison and Periodicity-Valued Comparison Functions
- JSON Functions
- Window Functions
- Sampling Analysis
- Configuring Indexes
- Reindexing
- Multi-Topic Search
- Context Search and Analysis
- Custom Redirect
- Downloading Log
- Search and Analysis (Metric Topic)
- Dashboard
- Data Processing documents
- Data Processing
- Data Processing Overview
- Creating Processing Task
- Viewing Data Processing Details
- Data Processing Functions
- Function Overview
- Key-Value Extraction Functions
- Enrichment Functions
- Flow Control
- Row Processing Functions
- Field Processing Functions
- Value Structuring Functions
- Regular Expression Processing Functions
- Time Value Processing Functions
- String Processing Functions
- Type Conversion Functions
- Logical and Mathematical Functions
- Encoding and Decoding Functions
- IP Parsing Functions
- Processing Cases
- Scheduled SQL Analysis
- SCF
- Data Processing
- Shipping and Consumption
- Monitoring Alarm
- Historical Documentation
- Practical Tutorial
- Developer Guide
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Topic Management APIs
- Log Set Management APIs
- Index APIs
- Topic Partition APIs
- Machine Group APIs
- Collection Configuration APIs
- Log APIs
- Metric APIs
- Alarm Policy APIs
- Data Processing APIs
- Kafka Protocol Consumption APIs
- CKafka Shipping Task APIs
- Kafka Data Subscription APIs
- COS Shipping Task APIs
- SCF Delivery Task APIs
- Scheduled SQL Analysis APIs
- COS Data Import Task APIs
- Data Types
- Error Codes
- FAQs
- CLS Service Level Agreement
- CLS Policy
- Contact Us
- Glossary
- Release Notes and Announcements
- User Guide
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Resource Management
- Permission Management
- Log Collection
- Collection Overview
- Collecting Logs in Self-Built Kubernetes Cluster
- Collecting Syslog
- Collection by LogListener
- Collecting Text Log
- Uploading Log over Kafka
- Uploading Logs via Anonymous Write
- Uploading Logs via Logback Appender
- Uploading Logs via Log4j Appender
- Uploading Log via SDK
- Uploading Log via API
- Importing Data
- Tencent Cloud Service Log Access
- Metric Collection
- Log Storage
- Metric Storage
- Search and Analysis (Log Topic)
- Overview of Search and Analysis
- Syntax and Rules
- Statistical Analysis (SQL)
- Quick Analysis
- SQL Syntax
- SQL Functions
- String Function
- Date and Time Functions
- IP Geographic Function
- URL Function
- Mathematical Calculation Functions
- Mathematical Statistical Function
- General Aggregate Function
- Geospatial Function
- Binary String Function
- Estimation Function
- Type Conversion Function
- Logical Function
- Operators
- Bitwise Operation
- Regular Expression Function
- Lambda Function
- Conditional Expressions
- Array Functions
- Interval-Valued Comparison and Periodicity-Valued Comparison Functions
- JSON Functions
- Window Functions
- Sampling Analysis
- Configuring Indexes
- Reindexing
- Multi-Topic Search
- Context Search and Analysis
- Custom Redirect
- Downloading Log
- Search and Analysis (Metric Topic)
- Dashboard
- Data Processing documents
- Data Processing
- Data Processing Overview
- Creating Processing Task
- Viewing Data Processing Details
- Data Processing Functions
- Function Overview
- Key-Value Extraction Functions
- Enrichment Functions
- Flow Control
- Row Processing Functions
- Field Processing Functions
- Value Structuring Functions
- Regular Expression Processing Functions
- Time Value Processing Functions
- String Processing Functions
- Type Conversion Functions
- Logical and Mathematical Functions
- Encoding and Decoding Functions
- IP Parsing Functions
- Processing Cases
- Scheduled SQL Analysis
- SCF
- Data Processing
- Shipping and Consumption
- Monitoring Alarm
- Historical Documentation
- Practical Tutorial
- Developer Guide
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Topic Management APIs
- Log Set Management APIs
- Index APIs
- Topic Partition APIs
- Machine Group APIs
- Collection Configuration APIs
- Log APIs
- Metric APIs
- Alarm Policy APIs
- Data Processing APIs
- Kafka Protocol Consumption APIs
- CKafka Shipping Task APIs
- Kafka Data Subscription APIs
- COS Shipping Task APIs
- SCF Delivery Task APIs
- Scheduled SQL Analysis APIs
- COS Data Import Task APIs
- Data Types
- Error Codes
- FAQs
- CLS Service Level Agreement
- CLS Policy
- Contact Us
- Glossary
Basic Concepts
Search and analysis support filtering, searching and statistically analyzing log data, such as querying logs containing
error, counting the number of logs by URL grouping, calculating PV change trend, etc. It is the most commonly used function in the Cloud Log Service.Search and analysis are based on Word Segmentation and Indexing. If you are unfamiliar with the two concepts, it is recommended that you consult this document in advance.
Features
Basic Features
Full-text search: Search the full text of logs. The log can be found when any field in the log meets the search conditions, such as using
error to search for all logs with errors.Key-value search: Search for the specified field in the log, and the log will be found when the specified field meets the search criteria. For example, use
level:error AND timeCost:>1000 to search for logs whose level is error and that consume time (timeCost) greater than 1,000 ms.Statistical analysis: You can perform statistical analysis on the logs meeting specified search criteria, and returns the results. For example, you can enter
status:404 | select count(*) as logCounts to get the number of logs whose response status code is 404.The above functions require the use of search and analysis statements. See Syntax Rules for details.
Note:
Infrequent storage only supports full-text search, not key-value retrieval or statistical analysis.
Advanced Features
Log Search
Multi-topic retrieval: Retrieve multiple log topics simultaneously, but statistical analysis is not supported. It can span log sets but not regions.
Contextual search: Retrieve several logs before (previous logs) or after (subsequent logs) the target log data in the original file. By viewing the context of specified logs, you can quickly find fault information during troubleshooting, making it easier to locate issues.
Custom redirect: Click the log field value to perform a custom redirect, facilitating further operations, such as querying user information on the internal user management platform based on user_id.
Download logs: Download logs that meet the search criteria to the local machine, with a maximum of 50 million logs per download.
Log statistical analysis
Rapid analysis: Click the log field name to quickly analyze the distribution of field values, trends over time, and numerical statistics.
Sampling analysis: When using the statistical analysis feature, if the number of original logs is very large or the statistical analysis statement (SQL) is complex, the analysis may be slow or even timeout. In this case, you can use the sampling analysis feature to randomly sample the original logs and then perform statistical analysis.
Associate external data: When performing statistical analysis on logs, you can associate the logs with external data for analysis. For example, you can query the user gender in the user information database based on the user ID in the logs, and then perform statistical analysis on the logs by gender.
Advantages
Based on pre-built index data, the search and analysis performance is much higher than that of queries with general text scanning methods (such as Linux grep commands and query functions in text editor), making the product suitable for real-time query of massive log data.
Statistical analysis supports SQL and conforms to the SQL-92 specification. In addition, flexible SQL syntax and 400+ SQL functions can be used for log statistics, meeting most statistical analysis requirements.
Fee Description
Search and analysis itself do not incur fees, but since it relies on reporting logs to the CLS platform and enabling indexing, it will generate log and index traffic fees, storage fees, and topic partition fees. Please see Billing Overview. Additionally, using the API to call the search and analysis interface will incur service request fees, and using the log download feature will incur public network read traffic fees.
Specifications and Limits
Yes
No
Was this page helpful?