tencent cloud

Feedback

Querying Cluster Logs

Last updated: 2024-08-16 19:04:49
    This document describes how to use ES cluster logs. You can use the logs to learn about the running status of the cluster, locate problems, and assist with cluster application development and OPS.

    Querying Cluster Logs

    1. Log in to the ES Console and click a cluster name to enter the cluster details page.
    2. Select the Cluster Logs page to view the logs of the cluster.
    ES has four types of logs: master log, slow search log, slow index log, and GC log. The log content includes log time, log level, and specific information.
    ES retains cluster logs of the last 7 days by default, which are displayed from newest to oldest and can be queried by time and keyword. In addition, you can also call ES APIs to adjust the log-related configuration. For example, for slow logs, you can set the time threshold that is considered as slow response during data querying or indexing.
    3. In the search box on the log page, you can query related logs by time range and keyword. The keyword query syntax is the same as the Lucene query syntax.
    Enter a keyword for querying, such as "YELLOW".
    Specify a field-specific keyword, such as "message:YELLOW".
    Combine multiple criteria for querying, such as level: INFO and ip: 10.0.1.1.
    

    Log Description

    Master log

    It displays the time, level, and information of the log generated by the cluster and has different levels such as INFO, WARN, and DEBUG.
    2019-2-14 08:00:00 10.0****
    INFO
    [o.e.c.r.a.AllocationService] [1550199698000783811] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[filebeat-2019.02.19][2]] ...]).
    
    2019-2-14 02:30:02 10.0****
    DEBUG
    [o.e.a.a.i.d.TransportDeleteIndexAction] [1550199698000783811] failed to delete indices [[[filebeat-2019.02.09/7VZM6Fa-Twmj8pVaAyyrxg]]]
    org.elasticsearch.index.IndexNotFoundException: no such index
    at org.elasticsearch.cluster.metadata.MetaData.getIndexSafe(MetaData.java:475) ~[elasticsearch-5.6.4.jar:5.6.4]
    at org.elasticsearch.cluster.metadata.MetaDataDeleteIndexService.lambda$deleteIndices$0(MetaDataDeleteIndexService.jav

    Slow log

    Slow log is used to capture querying and indexing requests that exceed the specified time threshold for tracking and analyzing very slow requests generated by user.
    2018-10-28 12:04:17
    WARN
    [index.indexing.slowlog.index] [1540298502000001009] [pmc/wCALr6BfRm-sr3qOQuGX
    Xw] took[18.6ms], took_millis[18], type[articles], id[AWa41-J9c0s1mOPvR6F3], routing[] , source[]
    Enable and adjust slow log Slow log is disabled by default. To enable it, you need to define the specific action (query, fetch, or index), expected event record level (INFO, WARN, or DEBUG), and time threshold. You can enable and adjust related configuration based on your business needs.
    To enable slow log, click Kibana in the top-right corner on the cluster details page to enter the Kibana page, call the Elasticsearch-related API via Dev Tools, or call the configuration modifying API through the client.
    Configure all indices:
    PUT */_settings
    {
    "index.indexing.slowlog.threshold.index.debug" : "5ms",
    "index.indexing.slowlog.threshold.index.info" : "50ms",
    "index.indexing.slowlog.threshold.index.warn" : "100ms",
    "index.search.slowlog.threshold.fetch.debug" : "10ms",
    "index.search.slowlog.threshold.fetch.info" : "50ms",
    "index.search.slowlog.threshold.fetch.warn" : "100ms",
    "index.search.slowlog.threshold.query.debug" : "100ms",
    "index.search.slowlog.threshold.query.info" : "200ms",
    "index.search.slowlog.threshold.query.warn" : "1s"
    }
    Configure one single index:
    PUT /my_index/_settings
    {
    "index.indexing.slowlog.threshold.index.debug" : "5ms",
    "index.indexing.slowlog.threshold.index.info" : "50ms",
    "index.indexing.slowlog.threshold.index.warn" : "100ms",
    "index.search.slowlog.threshold.fetch.debug" : "10ms",
    "index.search.slowlog.threshold.fetch.info" : "50ms",
    "index.search.slowlog.threshold.fetch.warn" : "100ms",
    "index.search.slowlog.threshold.query.debug" : "100ms",
    "index.search.slowlog.threshold.query.info" : "200ms",
    "index.search.slowlog.threshold.query.warn" : "1s"
    }

    GC log

    GC log is enabled in ES by default. The following two specific GC logs display the time, node IP, and log level of the log.
    2019-2-14 20:48:22
    10.0.***
    INFO
    [o.e.m.j.JvmGcMonitorService] [1550199698000783711] [gc][380573] overhead, spent [307ms] collecting in the last [1s]
    2019-2-14 10:04:09
    10.0.***
    WARN
    [o.e.m.j.JvmGcMonitorService] [1550199698000784111] [gc][341943] overhead, spent [561ms] collecting in the last [1s]
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support