tencent cloud

All product documents
Cloud Log Service
DocumentationCloud Log ServiceOperation GuideLog CollectionCollection by LogListenerLogListener Service Logs
LogListener Service Logs
Download PDF
Last updated: 2024-01-20 17:14:28
LogListener Service Logs
Last updated: 2024-01-20 17:14:28
Download PDF

Overview

LogListener service logs are used to record the operation, collection, and monitoring activities of LogListener, and you can configure visualized graphs to display such log data.

Default configuration

Default Configuration Item
Description
Log Topic
When you enable LogListener service logs, the logset cls_service_logging will be created for you automatically, and all log data generated by associated machine groups will be categorized and stored in corresponding log topics. The following three log topics are created by default:
loglistener_status: the heartbeat status logs for the corresponding LogListener.
loglistener_alarm: logs corresponding to LogListener's monitoring by collection metric/error type .
loglistener_business: logs corresponding to LogListener's collection operations, with each log corresponds to one request.
Region
After the LogListener log services are enabled, logsets and log topics will be created under the machine groups in the same region of LogListener.
Log Storage Duration
The default storage duration is 7 days, and the value cannot be modified.
Index
Full-text index and key-value index are enabled for all collected log data by default. You can modify the index configuration. For details, please see Configuring Index.
Dashboard
Dashboard service_log_dashboard will be created in the same region of LogListener by default.
Note:
LogListener service logs are only about collection and monitoring activities of LogListener, and do not support writing into other kinds of data.
Log data generated by this feature does not incur costs.
cls_service_logging is a unified logset for LogListener service logs.

Operations

Viewing LogListener status When the LogListener service logs are enabled, you can view LogListener running status and collection statistics. You can view the number of active LogListeners, LogListener status distribution and other statistical metrics on the service_log_dashboard dashboard.
Log collection monitoring configuration You can configure the collection and monitoring service logs by metric/error type. For example:
MEM, CPU, collection speed, collection latency, or other metrics.
The number of parsing errors of LogListener.
File-level monitoring With the LogListener service logs are enabled, you can view the monitoring logs of files and directories. For example:
All collection statistics files of one IP
The amount of logs collected under a certain path on a certain IP, such as app1 application logs located in /var/log/app1/. You can get the statistics of logs collected under this path.
The collection statistics of a topic.

Prerequisites

Only LogListener v2.5.4 and above support collection and monitoring service logs by machine/machine group. You’re advised to upgrade LogListener to the latest version.

Directions

Enabling the service logs

1. Log in to the CLS console.
2. In the left sidebar, click Machine Group to go to the machine group management page.
3. On the machine group list page, select the target machine group and click

to enable the LogListener service logs.

Disabling LogListener service logs

1. Log in to the CLS console.
2. In the left sidebar, click Machine Group to view the machine group list.
3. Select the target machine group, and click

to disable the LogListener service logs.
Note:
After this feature is disabled, the log data saved in the logset cls_service_logging will not be deleted automatically. You can manually delete the logset where the service logs are saved.

Dashboard of Service Logs

When the LogListener service logs are enabled, CLS will create a dashboard service_log_dashboard by the type of recorded logs to display LogListener’s collection and monitoring statistics.

Collection statistics dashboard

You can go to the Dashboard page of the CLS console, click the ID of the target dashboard to view LogListener collection statistics, including its status, parsing failure rate, sending success rate, and other metrics.

Log Types

LogListener status logs

The parameters of the log topic loglistener_status are detailed as follows:
Parameter
Description
InstanceId
LogListener unique identifier
IP
Machine group IP
Label
An array of machine IDs
Version
Version number
MemoryUsed
Memory utilization of LogListener
MemMax
Memory utilization threshold on this machine set by the Agent
CpuUsage
LogListener CPU utilization
Status
LogListener running status
TotalSendLogSize
Size of logs sent
SendSuccessLogSize
Size of successfully sent logs
SendFailureLogSize
Size of sending-failed logs
SendTimeoutLogSize
Size of logs with sending timed out
TotalParseLogCount
Total number of logs parsed
ParseFailureLogCount
Number of parsing-failed logs
TotalSendLogCount
Number of logs sent
SendSuccessLogCount
Number of successful sent logs
SendFailureLogCount
Number of sending-failed logs
SendTimeoutLogCount
Number of logs with sending timed out
TotalSendReqs
Total number of requests sent
SendSuccessReqs
Number of successful sent requests
SendFailureReqs
The number of sending-failed requests
SendTimeoutReqs
Number of requests with sending timed out
TotalFinishRsps
Total number of RSP files received
TotalSuccessFromStart
Total number of successfully sent requests since LogListener was enabled
AvgReqSize
Average request packet size
SendAvgCost
Average sending time
AvailConnNum
Number of available connections
QueueSize
The size of queued requests

LogListener alarm logs

The parameters of the log topic loglistener_alarm are detailed as follows:
Monitoring Metric
Description
InstanceId
LogListener unique identifier
Label
An array of machine IDs
IP
Machine group IP
Version
LogListener version
AlarmType.count
Statistics of alarm types
AlarmType.example
Sample alarm type
AlarmType:
alarm type
type ID
Description
UnknownError
0
Initializing the alarm type.
UnknownError
1
Failed to parse.
CredInvalid
2
Failed to verify.
SendFailure
3
Failed to send.
RunException
4
Abnormal LogListener running.
MemLimited
5
Reached the memory utilization threshold.
FileProcException
6
Exceptions occurred in file processing.
FilePosGetError
7
Failed to get the file publishing info.
HostIpException
8
Exceptions occurred in the server IP thread.
StatException
9
Failed to get the process info.
UpdateException
10
Exceptions occurred in the CLS modification feature.
DoSendError
11
Failed to confirm sending.
FileAddError
12
Failed to create the file.
FileMetaError
13
Failed to create the metadata file.
FileOpenError
14
Failed to open the file.
FileOpenError
15
Failed to read the file.
FileStatError
16
Failed to get the file status.
getTimeError
17
Failed to get the time from the log content.
HandleEventError
18
Exceptions occurred in processing the file.
handleFileCreateError
19
Exceptions occurred in handleFileCreateEvent().
LineParseError
20
Failed to parse the log directory.
Lz4CompressError
21
Failed to compress.
readEventException
22
Failed to read.
ReadFileBugOn
23
A bug exists.
ReadFileException
24
Exceptions occurred in the read file.
ReadFileInodeChange
25
File node changed.
ReadFileTruncate
26
The read file is truncated.
WildCardPathException
27
Exceptions occurred in addWildcardPathInotify().

LogListener collection logs

The parameters of the log topic loglistener_business are detailed as follows:
Parameter
Description
InstanceId
LogListener unique identifier
Label
An array of machine IDs
IP
Machine group IP
Version
LogListener version
TopicId
The target topic of the collected file
FileName
File path name
FileName
Actual file path
FileInode
File node
FileSize
File size
LastReadTime
The most recent read time of the file
ParseFailLines
Number of parsing-failed logs within a time window
ParseFailSize
Size of parsing-failed logs within a time window
ParseSuccessLines
Number of logs successful parsed within a time window
ParseSuccessSize
Size of logs successful parsed within a time window
ReadOffset
Offset of file reading in bytes
TruncateSize
Size of truncated log files within a time window
ReadAvgDelay
Average time delay for reads within a time window
TimeFormatFailuresLines
Number of timestamp matching errors within a time window
SendSuccessSize
Size of logs successful sent within a time window
SendSuccessCount
Number of logs successful sent within a time window
SendFailureSize
Size of sending-failed logs within a time window
SendFailureCount
Number of sending-failed logs within a time window
SendTimeoutSize
Size of logs with sending timed out within a time window
SendTimeoutCount
Number of logs with sending timed out within a time window
DroppedLogSize
Size of dropped logs within a time window
DroppedLogCount
Number of dropped logs in a time window
ProcessBlock
Whether the current file has triggered collection blocking in a statistical period (collection blocking will be triggered if the sliding window of a file has not moved for 10 minutes)

Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon