LogListener service logs are used to record the operation, collection, and monitoring activities of LogListener, and you can configure visualized graphs to display such log data.
Default configuration
Default Configuration Item
Description
Log Topic
When you enable LogListener service logs, the logset cls_service_logging will be created for you automatically, and all log data generated by associated machine groups will be categorized and stored in corresponding log topics. The following three log topics are created by default:
loglistener_status: the heartbeat status logs for the corresponding LogListener.
loglistener_alarm: logs corresponding to LogListener's monitoring by collection metric/error type .
loglistener_business: logs corresponding to LogListener's collection operations, with each log corresponds to one request.
Region
After the LogListener log services are enabled, logsets and log topics will be created under the machine groups in the same region of LogListener.
Log Storage Duration
The default storage duration is 7 days, and the value cannot be modified.
Index
Full-text index and key-value index are enabled for all collected log data by default. You can modify the index configuration. For details, please see Configuring Index.
Dashboard
Dashboard service_log_dashboard will be created in the same region of LogListener by default.
Note:
LogListener service logs are only about collection and monitoring activities of LogListener, and do not support writing into other kinds of data.
Log data generated by this feature does not incur costs.
cls_service_logging is a unified logset for LogListener service logs.
Operations
Viewing LogListener status
When the LogListener service logs are enabled, you can view LogListener running status and collection statistics. You can view the number of active LogListeners, LogListener status distribution and other statistical metrics on the service_log_dashboard dashboard.
Log collection monitoring configuration
You can configure the collection and monitoring service logs by metric/error type. For example:
MEM, CPU, collection speed, collection latency, or other metrics.
The number of parsing errors of LogListener.
File-level monitoring
With the LogListener service logs are enabled, you can view the monitoring logs of files and directories. For example:
All collection statistics files of one IP
The amount of logs collected under a certain path on a certain IP, such as app1 application logs located in /var/log/app1/. You can get the statistics of logs collected under this path.
The collection statistics of a topic.
Prerequisites
Only LogListener v2.5.4 and above support collection and monitoring service logs by machine/machine group. You’re advised to upgrade LogListener to the latest version.
2. In the left sidebar, click Machine Group to view the machine group list.
3. Select the target machine group, and click
to disable the LogListener service logs.
Note:
After this feature is disabled, the log data saved in the logset cls_service_logging will not be deleted automatically. You can manually delete the logset where the service logs are saved.
Dashboard of Service Logs
When the LogListener service logs are enabled, CLS will create a dashboard service_log_dashboard by the type of recorded logs to display LogListener’s collection and monitoring statistics.
Collection statistics dashboard
You can go to the Dashboard page of the CLS console, click the ID of the target dashboard to view LogListener collection statistics, including its status, parsing failure rate, sending success rate, and other metrics.
Log Types
LogListener status logs
The parameters of the log topic loglistener_status are detailed as follows:
Parameter
Description
InstanceId
LogListener unique identifier
IP
Machine group IP
Label
An array of machine IDs
Version
Version number
MemoryUsed
Memory utilization of LogListener
MemMax
Memory utilization threshold on this machine set by the Agent
CpuUsage
LogListener CPU utilization
Status
LogListener running status
TotalSendLogSize
Size of logs sent
SendSuccessLogSize
Size of successfully sent logs
SendFailureLogSize
Size of sending-failed logs
SendTimeoutLogSize
Size of logs with sending timed out
TotalParseLogCount
Total number of logs parsed
ParseFailureLogCount
Number of parsing-failed logs
TotalSendLogCount
Number of logs sent
SendSuccessLogCount
Number of successful sent logs
SendFailureLogCount
Number of sending-failed logs
SendTimeoutLogCount
Number of logs with sending timed out
TotalSendReqs
Total number of requests sent
SendSuccessReqs
Number of successful sent requests
SendFailureReqs
The number of sending-failed requests
SendTimeoutReqs
Number of requests with sending timed out
TotalFinishRsps
Total number of RSP files received
TotalSuccessFromStart
Total number of successfully sent requests since LogListener was enabled
AvgReqSize
Average request packet size
SendAvgCost
Average sending time
AvailConnNum
Number of available connections
QueueSize
The size of queued requests
LogListener alarm logs
The parameters of the log topic loglistener_alarm are detailed as follows:
Monitoring Metric
Description
InstanceId
LogListener unique identifier
Label
An array of machine IDs
IP
Machine group IP
Version
LogListener version
AlarmType.count
Statistics of alarm types
AlarmType.example
Sample alarm type
AlarmType:
alarm type
type ID
Description
UnknownError
0
Initializing the alarm type.
UnknownError
1
Failed to parse.
CredInvalid
2
Failed to verify.
SendFailure
3
Failed to send.
RunException
4
Abnormal LogListener running.
MemLimited
5
Reached the memory utilization threshold.
FileProcException
6
Exceptions occurred in file processing.
FilePosGetError
7
Failed to get the file publishing info.
HostIpException
8
Exceptions occurred in the server IP thread.
StatException
9
Failed to get the process info.
UpdateException
10
Exceptions occurred in the CLS modification feature.
DoSendError
11
Failed to confirm sending.
FileAddError
12
Failed to create the file.
FileMetaError
13
Failed to create the metadata file.
FileOpenError
14
Failed to open the file.
FileOpenError
15
Failed to read the file.
FileStatError
16
Failed to get the file status.
getTimeError
17
Failed to get the time from the log content.
HandleEventError
18
Exceptions occurred in processing the file.
handleFileCreateError
19
Exceptions occurred in handleFileCreateEvent().
LineParseError
20
Failed to parse the log directory.
Lz4CompressError
21
Failed to compress.
readEventException
22
Failed to read.
ReadFileBugOn
23
A bug exists.
ReadFileException
24
Exceptions occurred in the read file.
ReadFileInodeChange
25
File node changed.
ReadFileTruncate
26
The read file is truncated.
WildCardPathException
27
Exceptions occurred in addWildcardPathInotify().
LogListener collection logs
The parameters of the log topic loglistener_business are detailed as follows:
Parameter
Description
InstanceId
LogListener unique identifier
Label
An array of machine IDs
IP
Machine group IP
Version
LogListener version
TopicId
The target topic of the collected file
FileName
File path name
FileName
Actual file path
FileInode
File node
FileSize
File size
LastReadTime
The most recent read time of the file
ParseFailLines
Number of parsing-failed logs within a time window
ParseFailSize
Size of parsing-failed logs within a time window
ParseSuccessLines
Number of logs successful parsed within a time window
ParseSuccessSize
Size of logs successful parsed within a time window
ReadOffset
Offset of file reading in bytes
TruncateSize
Size of truncated log files within a time window
ReadAvgDelay
Average time delay for reads within a time window
TimeFormatFailuresLines
Number of timestamp matching errors within a time window
SendSuccessSize
Size of logs successful sent within a time window
SendSuccessCount
Number of logs successful sent within a time window
SendFailureSize
Size of sending-failed logs within a time window
SendFailureCount
Number of sending-failed logs within a time window
SendTimeoutSize
Size of logs with sending timed out within a time window
SendTimeoutCount
Number of logs with sending timed out within a time window
DroppedLogSize
Size of dropped logs within a time window
DroppedLogCount
Number of dropped logs in a time window
ProcessBlock
Whether the current file has triggered collection blocking in a statistical period (collection blocking will be triggered if the sliding window of a file has not moved for 10 minutes)
Yes
No
Was this page helpful?