tencent cloud

All product documents
Elastic MapReduce
DocumentationElastic MapReduce EMR on CVM Operation GuideMonitoring and AlarmsLog
Log
Download PDF
Last updated: 2024-01-10 09:56:47
Log
Last updated: 2024-01-10 09:56:47
Download PDF

Overview

The log search feature collects component operation logs. Then, you can search for core service logs of the current cluster and node system logs by keyword to quickly view key service logs without logging in to nodes.

Directions

1. Log in to the EMR console and click the ID/Name of the target cluster in the cluster list.
2. On the cluster details page, choose Cluster Monitoring > Log Search to filter and view log content by current cluster, log file, node IP, and time range. Alternatively, choose Cluster Service > target component block > Role Management, click a Node IP in Role List to enter the page where the node monitoring metrics are displayed, and then click Role Log to enter the Log Search page.

Click a Node IP to enter the Node Status page, or click Log Source to enter the page where the node monitoring metrics are displayed.

Keyword description:
Supports full-text search based on keywords.
Supports search by using special symbols, including hyphen (-), period (.), asterisk (*), greater-than sign (>), less-than sign (<), equal sign (=), exclamation mark (!), brackets (() and {}), and forward slash (/).
Supports phrase-based search. Example: address=/ip:port.
3. During troubleshooting, you usually need to pay attention to contextual logs of keywords. On the Log Search page, click View Context to enter the Log Context page.
Note:
Currently, only logs in the past 15 days can be searched for.
If log collection has not been enabled on your cluster, you can contact your Tencent Cloud rep for assistance.
Component
Role
Log
Description
HDFS
NameNode
/data/emr/hdfs/logs/hadoop-hadoop-namenode.log
NameNode runtime log
ZKFC
/data/emr/hdfs/logs/hadoop-hadoop-zkfc.log
ZKFC runtime log
DataNode
/data/emr/hdfs/logs/hadoop-hadoop-datanode.log
DataNode runtime log
JournalNode
/data/emr/hdfs/logs/hadoop-hadoop-journalnode.log
JournalNode runtime log
DFSRouter
/data/emr/hdfs/logs/hadoop-hadoop-dfsrouter.log
DFRouter runtime log
YARN
ResourceManager
/data/emr/yarn/logs/yarn-hadoop-resourcemanager.log
ResourceManager runtime log
NodeManager
/data/emr/yarn/logs/yarn-hadoop-nodemanager.log
NodeManager runtime log
JobHistoryServer
/data/emr/yarn/logs/mapred-hadoop-historyserver.log
JobHistoryServer runtime log
HBase
HMaster
/data/emr/hbase/logs/hbase-hadoop-master.log
HMaster runtime log
ThriftServer
/data/emr/hbase/logs/hbase-hadoop-thrift.log
ThriftServer runtime log
RegionServer
/data/emr/hbase/logs/hbase-hadoop-regionserver.log
RegionServer runtime log
ClickHouse
ClickHouse-server
/data/clickhouse/clickhouse-server/logs/clickhouse-server.log
ClickHouse-server runtime log
Druid
Broker
/data/emr/druid/var/log/druid/broker.log
Broker runtime log
Coordinator
/data/emr/druid/var/log/druid/coordinator.log
Coordinator runtime log
Router
/data/emr/druid/var/log/druid/router.log
Router runtime log
Overload
/data/emr/druid/var/log/druid/overload.log
Overload runtime log
Historical
/data/emr/druid/var/log/druid/historical.log
Historical runtime log
MiddleManager
/data/emr/druid/var/log/druid/middleManager.log
MiddleManager runtime log
Zookeeper
Zookeeper
/data/emr/zookeeper/logs/zookeeper-root-server.log
ZooKeeper runtime log
Hive
HiveServer2
/data/emr/hive/logs/hadoop-hive
HiveServer2 runtime log
Kudu
KuduMaster
/data/emr/kudu/logs/kudu-master.WARNING
KuduMaster runtime log
KuduServer
/data/emr/kudu/logs/kudu-tserver.WARNING
KuduServer runtime log
Alluxio
AlluxioMaster
/data/emr/alluxio/logs/master.log
AlluxioMaster runtime log
AlluxioWorker
/data/emr/alluxio/logs/worker.log
AlluxioWorker runtime log
Ranger
EmbeddedServer
/data/emr/ranger/logs/ranger-admin.log
EmbeddedServer runtime log
CosRanger
CosRangerServer
/usr/local/service/cosranger/log/info.log
COSRanger runtime log
Impala
Catalogd
/data/emr/impala/logs/catalogd.INFO
Cataloged runtime log
Statestored
/data/emr/impala/logs/statestored.INFO
Statestored runtime log
Impalad
/data/emr/impala/logs/impalad.INFO
Impalad runtime log
Spark
HistoryServer
/data/emr/spark/logs/spark-hadoop.log
HistoryServer runtime log
Kylin
Kylin
/data/emr/kylin/logs/kylin.log
Kylin runtime log
Zeppelin
ZeppelinServer
/data/emr/zepplin/logs/zeppelin-hadoop.log
ZeppelinServer runtime log
Knox
Gateway
/data/emr/knox/logs/gateway.log
Gateway runtime log
Doris
BrokerBootstrap
/data/emr/doris/broker/log/apache_hdfs_broker.log
BrokerBootstrap runtime log
PaloFe
/data/emr/doris/fe/log/fe.log
PaloFe runtime log
PaloBe
/data/emr/doris/be/log/be.INFO
PaloBe runtime log
Kafka
Kafka
/user/local/service/kafka/logs/server.log
Kafka runtime log

Minimum Log Level Supported by Services

Service
Default Minimum Log Level
Impala and Kudu
INFO
Other services
WARN

Query Rules for the Minimum Log Level

Minimum Log Level
Queryable Log Levels
INFO
INFO, WARN, ERROR, FATAL
WARN
WARN, ERROR, FATAL
ERROR
ERROR, FATAL
FATAL
FATAL

Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 available.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon