tencent cloud

All product documents
Tencent Cloud Observability Platform
DocumentationTencent Cloud Observability PlatformCloud Product MonitoringTencent Cloud Service MetricsData AnalysisEMREMR (HDFS)
EMR (HDFS)
Download PDF
Last updated: 2024-01-27 17:54:54
EMR (HDFS)
Last updated: 2024-01-27 17:54:54
Download PDF

Namespace

Namespace=QCE/TXMR_HDFS

Monitoring Metrics

Note:
For more information on the parameters in each dimension, please see Overview of the Parameters in Each Dimension.

HDFS - Overview

Parameter
Metric Name
Unit
Description
Dimension
EmrHdfsOverview HdfsNnBlockCapacityTotal
Cluster storage capacity_CapacityTotal
GB
Total cluster storage capacity
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockCapacityUsed
Cluster storage capacity_CapacityUsed
GB
Used cluster storage capacity
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockCapacityRemaining
Cluster storage capacity_CapacityRemaining
GB
Remaining cluster storage capacity
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockCapacity UsedNonDFS
Cluster storage capacity_CapacityUsedNonDFS
GB
Non-HDFS used cluster capacity
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockTotalLoad
Cluster load_TotalLoad
Count
Number of current connections
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockFilesTotal
Total number of cluster files_FilesTotal
Count
Total number of files
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockBlockstotal
Number of blocks_BlocksTotal
Count
Total number of blocks
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockPending ReplicationBlocks
Number of blocks_PendingReplicationBlocks
Count
Number of blocks waiting to be backed up
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockUnder ReplicatedBlocks
Number of blocks_UnderReplicatedBlocks
Count
Number of blocks with insufficient replicas
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockBlocksCorruptblocks
Number of blocks_CorruptBlocks
Count
Number of corrupted blocks
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockScheduled ReplicationBlocks
Number of blocks_ScheduledReplicationBlocks
Count
Number of blocks arranged for backup
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockPending DeletionBlocks
Number of blocks_PendingDeletionBlocks
Count
Number of blocks waiting to be deleted
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockCorruptblocks
Number of blocks_CorruptBlocks
Count
Number of excessive blocks
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockPostponed MisreplicatedBlocks
Number of blocks_PostponedMisreplicatedBlocks
Count
Number of blocks with exceptions that were postponed to be processed
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockBlockCapacity
Capacity of blocks_BlockCapacity
Count
Capacity of blocks
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNumLiveDataNodes
Cluster DataNodes_NumLiveDataNodes
Count
Number of live DataNodes
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNumDeadDataNodes
Cluster DataNodes_NumDeadDataNodes
Count
Number of DataNodes marked as dead
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNum DecomLiveDataNodes
Cluster DataNodes_NumDecomLiveDataNodes
Count
Number of decommissioned live nodes
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNum DecomDeadDataNodes
Cluster DataNodes_NumDecomDeadDataNodes
Count
Number of decommissioned dead nodes
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNum DecommissioningDataNodes
Cluster DataNodes_NumDecommissioningDataNodes
Count
Number of decommissioning nodes
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockNum StaleDataNodes
Cluster DataNodes_NumStaleDataNodes
Count
Number of current DataNodes marked as expired due to heartbeat delay
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockSnapshots
Snapshots_Snapshots
Count
Number of snapshots
host4hdfsoverview, id4hdfsoverview
EmrHdfsOverview HdfsNnBlockVolumeFailuresTotal
Disk failures_VolumeFailuresTotal
Count
Total number of failures on all DataNodes
host4hdfsoverview, id4hdfsoverview

HDFS - OverviewAggregation

Parameter
Metric Name
Unit
Description
Dimension
EmrHdfsOverview Aggregation HdfsNnBlockCapacityTotal
Cluster storage capacity_CapacityTotal
GB
Total cluster storage capacity
id4hdfsoverview
EmrHdfsOverview Aggregation HdfsNnBlockCapacityUsed
Cluster storage capacity_CapacityUsed
GB
Used cluster storage capacity
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockCapacityRemaining
Cluster storage capacity_CapacityRemaining
GB
Remaining cluster storage capacity
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockCapacity UsedNonDFS
Cluster storage capacity_CapacityUsedNonDFS
GB
Non-HDFS used cluster capacity
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockTotalLoad
Cluster load_TotalLoad
Count
Number of current connections
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockFilesTotal
Total number of cluster files_FilesTotal
Count
Total number of files
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockBlockstotal
Number of blocks_BlocksTotal
Count
Total number of blocks
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockPending
ReplicationBlocks
Number of blocks_PendingReplicationBlocks
Count
Number of blocks waiting to be backed up
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockUnder
ReplicatedBlocks
Number of blocks_UnderReplicatedBlocks
Count
Number of blocks with insufficient replicas
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockBlocksCorruptblocks
Number of blocks_CorruptBlocks
Count
Number of corrupted blocks
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockScheduled ReplicationBlocks
Number of blocks_ScheduledReplicationBlocks
Count
Number of blocks arranged for backup
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockPending
DeletionBlocks
Number of blocks_PendingDeletionBlocks
Count
Number of blocks waiting to be deleted
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockCorruptblocks
Number of blocks_CorruptBlocks
Count
Number of excessive blocks
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockPostponed MisreplicatedBlocks
Number of blocks_PostponedMisreplicatedBlocks
Count
Number of blocks with exceptions that were postponed to be processed
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockBlockCapacity
Capacity of blocks_BlockCapacity
Count
Capacity of blocks
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNumLiveDataNodes
Cluster DataNodes_NumLiveDataNodes
Count
Number of live DataNodes
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNumDeadDataNodes
Cluster DataNodes_NumDeadDataNodes
Count
Number of DataNodes marked as dead
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNum DecomLiveDataNodes
Cluster DataNodes_NumDecomLiveDataNodes
Count
Number of decommissioned live nodes
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNum DecomDeadDataNodes
Cluster DataNodes_NumDecomDeadDataNodes
Count
Number of decommissioned dead nodes
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNum DecommissioningDataNodes
Cluster DataNodes_NumDecommissioningDataNodes
Count
Number of decommissioning nodes
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockNum StaleDataNodes
Cluster DataNodes_NumStaleDataNodes
Count
Number of current DataNodes marked as expired due to heartbeat delay
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockSnapshots
Snapshots_Snapshots
Count
Number of snapshots
id4hdfsoverview
EmrHdfsOverview AggregationHdfsNn BlockVolumeFailuresTotal
Disk failures_VolumeFailuresTotal
Count
Total number of failures on all DataNodes
id4hdfsoverview

HDFS - NameNode

Parameter
Metric Name
Unit
Description
Dimension
HdfsNnPort4007RxtxReceivedbytes
Data traffic_ReceivedBytes
Bytes/s
Data receiving rate
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007RxtxSentbytes
Data traffic_SentBytes
Bytes/s
Data sending rate
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007Qps Rpcqueuetimenumops
QPS_RpcQueueTimeNumOps
Calls/s
RPC call rate
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007RtRpc queuetimeavgtime
Request processing delay
_RpcQueueTimeAvgTime
ms
Average RPC delay
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007AuthRpc authenticationfailures
Authentication and authorization
_RpcAuthenticationFailure
Count
Number of RPC authentication failures
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007AuthRpc authenticationsuccesses
Authentication and authorization
_RpcAuthenticationSuccesses
Count
Number of RPC authentication successes
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007AuthRpc authorizationfailures
Authentication and authorization
_RpcAuthorizationFailures
Count
Number of RPC authorization failures
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007AuthRpc authorizationsuccesses
Authentication and authorization
_RpcAuthorizationSuccesses
Count
Number of RPC authorization successes
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007Connections Numopenconnections
Number of current connections
_NumOpenConnections
Count
Number of current connections
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPort4007Queue LenCallqueuelength
RPC processing queue length
_CallQueueLength
Count
Length of current RPC processing queue
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMemMemnonheapusedm
JVM memory
_MemNonHeapUsedM
MB
Size of the NonHeapMemory currently used by JVM
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMemMemnon heapcommittedm
JVM memory
_MemNonHeapCommittedM
MB
JVM memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMemMemheapusedm
JVM memory
_MemHeapUsedM
MB
Size of the HeapMemory currently used by JVM
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMem Memheapcommittedm
JVM memory
_MemHeapCommittedM
MB
Size of the HeapMemory committed by JVM
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMemMemheapmaxm
JVM memory_MemHeapMaxM
MB
Size of the HeapMemory configured by JVM
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmMemMemmaxm
JVM memory_MemMaxM
MB
Maximum memory size that can be used by JVM during runtime
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlockReportRt Blockreportavgtime
Block reporting delay
_BlockReportAvgTime
Blocks/s
Average delay in processing DataNode blocks per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilGcCountFgc
GC count_FGC
Operations/s
Full GC count
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilGcCountYgc
GC count_YGC
2 operations/s
Young GC count
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilGcTimeYgct
GC time_YGCT
ms
Time consumed by Young GC
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilGcTimeFgct
GC time_FGCT
ms
Time consumed by Full GC
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilGcTimeGct
GC time_GCT
ms
Time used to collect garbage
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreadsThreadsnew
Number of JVM threads
_ThreadsNew
Count
Number of threads in the new state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreads Threadsrunnable
Number of JVM threads
_ThreadsRunnable
Count
Number of threads in the runnable state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreads Threadsblocked
Number of JVM threads
_ThreadsBlocked
Count
Number of threads in the blocked state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreads Threadswaiting
Number of JVM threads
_ThreadsWaiting
Count
Number of threads in the WAITING state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreads Threadstimedwaiting
Number of JVM threads
_ThreadsTimedWaiting
Count
Number of threads in the TIMED WAITING state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmJavaThreads Threadsterminated
Number of JVM threads
_ThreadsTerminated
Count
Number of threads in the Terminated state
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmLogTotalLogfatal
Number of JVM logs_LogFatal
Count
Number of Fatal logs
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmLogTotalLogerror
Number of JVM logs_LogError
Count
Number of Error logs
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmLogTotalLogwarn
Number of JVM logs_LogWarn
Count
Number of Warn logs
host4hdfsnamenode, id4hdfsnamenode
HdfsNnJvmLogTotalLoginfo
Number of JVM logs_LogInfo
Count
Number of Info logs
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryS0
Memory space percentage_S0
%
Percentage of used Survivor 0 memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryS1
Memory space percentage_S1
%
Percentage of used Survivor 1 memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryE
Memory space percentage_E
%
Percentage of used Eden memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryO
Memory space percentage_O
%
Percentage of used Old memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryM
Memory space percentage_M
%
Percentage of used Metaspace memory
host4hdfsnamenode, id4hdfsnamenode
HdfsNnGcUtilMemoryCcs
Memory space percentage_CCS
%
Percentage of memory used by compressed class space
host4hdfsnamenode, id4hdfsnamenode
HdfsNnStaleStorages CountNumstalestorages
Number of storages marked as expired
_NumStaleStorages
Count
Number of current DataNodes marked as expired due to heartbeat delay
host4hdfsnamenode, id4hdfsnamenode
HdfsNnPendingDatanodeMessage CountPendingdatanode messagecount
Number of pending block operation messages on the standby NameNode
_PendingDataNode
MessageCount
Requests/s
Number of DataNode requests queued on the standby NameNode
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlocksMissingNum berofmissingblocks
Total number of missing blocks
_NumberOfMissingBlocks
Count
Number of missing data blocks
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlocksMissingNumberof missingblockswithreplication factorOne
Total number of missing blocks_NumberOf
MissingBlocksWithReplication
FactorOne
Count
Number of missing blocks (rf = 1)
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOpsAllowsnapshotops
Snapshot operations
_AllowSnapshotOps
Operations/s
Number of AllowSnapshot operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Disallowsnapshotops
Snapshot operations
_DisallowSnapshotOps
Operations/s
Number of DisallowSnapshot operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Createsnapshotops
Snapshot operations
_CreateSnapshotOps
Operations/s
Number of CreateSnapshot operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Deletesnapshotops
Snapshot operations
_DeleteSnapshotOps
Operations/s
Number of DeleteSnapshot operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Listsnapshottabledirops
Snapshot operations_ListSnapshottableDirOps
Operations/s
Number of ListSnapshottableDir operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Snapshotdiffreportops
Snapshot operations_SnapshotDiffReportOps
Operations/s
Number of SnapshotDiffReportOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSnapshotOps Renamesnapshotops
Snapshot operations_RenameSnapshotOps
Operations/s
Number of RenameSnapshotOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsCreatefileops
File operations_CreateFileOps
Operations/s
Number of CreateFile operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsGetlistingops
File operations_GetListingOps
Operations/s
Number of GetListing operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsTotalfileops
File operations_TotalFileOps
Operations/s
Number of TotalFileOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsDeletefileops
File operations_DeleteFileOps
Operations/s
Number of DeleteFile operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsFileinfoops
File operations_FileInfoOps
Operations/s
Number of FileInfo operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsGetadditional datanodeops
File operations
_GetAdditionalDatanodeOps
Operations/s
Number of GetAdditionalDatanode operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsCreatesymlinkops
File operations_CreateSymlinkOps
Operations/s
Number of CreateSymlink operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsGetlinktargetops
File operations_GetLinkTargetOps
Operations/s
Number of GetLinkTarget operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnFilesOpsFilesingetlistingops
File operations
_FilesInGetListingOps
Operations/s
Number of FilesInGetListing operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnTransactionOps Transactionsnumops
Transaction operations
_TransactionsNumOps
Operations/s
Number of Journal transaction operations processed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnTransactionOps Transactionsbatchedinsync
Transaction operations
_TransactionsBatchedInSync
Operations/s
Number of Journal transaction operations processed in batches per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageOpsGeteditnumops
Image operations_GetEditNumOps
Operations/s
Number of GetEditNumOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageOpsGetimagenumops
Image operations_GetImageNumOps
Operations/s
Number of GetImageNumOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageOpsPutimagenumops
Image operations_PutImageNumOps
Operations/s
Number of PutImageNumOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSyncsOpsSyncsnumops
Sync operations_SyncsNumOps
Operations/s
Number of Journal syncs operations processed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlocksOpsBlock receivedanddeletedops
Data block operations
_BlockOpsQueued
Operations/s
Number of BlockReceivedAndDeletedOps operations executed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlocksOpsBlockopsqueued
Data block operations
_BlockOpsQueued
Operations/s
Delay in processing DataNode block reporting operations
host4hdfsnamenode, id4hdfsnamenode
HdfsNnCacheReportOps Cachereportnumops
Cache reporting
_CacheReportNumOps
Operations/s
Number of CacheReport operations processed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnBlockReportOps Blockreportnumops
Block reporting
_BlockReportNumOps
Operations/s
Number of DataNode block reporting operations processed per second
host4hdfsnamenode, id4hdfsnamenode
HdfsNnSyncsRtSyncsavgtime
Sync operation delay_SyncsAvgTime
ms
Average delay in processing Journal syncs operations
host4hdfsnamenode, id4hdfsnamenode
HdfsNnCacheReportRt Cachereportavgtime
Cache reporting delay_CacheReportAvgTime
ms
Average delay in cache reporting operations
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageRtGeteditavgtime
Image operation delay
_GetEditAvgTime
ms
Average delay in reading the Edit file
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageRtGetimageavgtime
Image operation delay
_GetImageAvgTime
ms
Average delay in reading the image file
host4hdfsnamenode, id4hdfsnamenode
HdfsNnImageRtPutimageavgtime
Image operation delay
_PutImageAvgTime
ms
Average delay in writing the image file
host4hdfsnamenode, id4hdfsnamenode
HdfsNnTransactionRt Transactionsavgtime
Transaction operation delay
_TransactionsAvgTime
ms
Average delay in processing Journal transaction operations
host4hdfsnamenode, id4hdfsnamenode
HdfsNnStartTimeStarttime
Start time_StartTime
ms
Process start time
host4hdfsnamenode, id4hdfsnamenode
HdfsNnStateState
Primary/secondary status_State
-
NN status
host4hdfsnamenode, id4hdfsnamenode
HdfsNnThreadCountPeakthreadcount
Number of threads_PeakThreadCount
Count
Peak number of threads
host4hdfsnamenode, id4hdfsnamenode
HdfsNnThreadCountThreadcount
Number of threads_ThreadCount
Count
Number of threads
host4hdfsnamenode, id4hdfsnamenode
HdfsNnThreadCount Daemonthreadcount
Number of threads_DaemonThreadCount
Count
Number of background threads
host4hdfsnamenode, id4hdfsnamenode

HDFS - DataNode

Parameter
Metric Name
Unit
Description
Dimension
HdfsDnXceiverXceivercount
Number of Xceivers_XceiverCount
Xceivers
Number of Xceivers
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBytesByteswrittenmb
Data read/write rate_BytesReadMB
Bytes/s
DN byte write rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBytesBytesreadmb
Data read/write rate_BytesReadMB
Bytes/s
DN byte read rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBytesRemotebytesreadmb
Data read/write rate _RemoteBytesReadMB
Bytes/s
Rate of bytes read by the remote client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBytesRemoteby teswrittenmb
Data read/write rate _RemoteBytesWrittenMB
Bytes/s
Rate of bytes written by the remote client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnClientWritesfrom remoteclient
Number of client connections _WritesFromRemoteClient
Count
QPS of write operations from the remote client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnClientWritesfromlocalclient
Number of client connections _WritesFromLocalClient
Count
OPS of write operations from the local client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnClientReadsfrom remoteclient
Number of client connections _ReadsFromRemoteClient
Count
QPS of read operations from the remote client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnClientReadsfromlocalclient
Number of client connections _ReadsFromLocalClient
Count
QPS of read operations from the local client
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksVerifiedFailures Blockverificationfailures
Block check failure _BlockVerificationFailures
Failures/s
Number of block check failures
id4hdfsdatanode, host4hdfsdatanode
HdfsDnVolumeFailures Volumefailures
Disk failures_VolumeFailures
Failures/s
Number of disk failures
id4hdfsdatanode, host4hdfsdatanode
HdfsDnNetworkErrors Datanodenetworkerrors
Network errors _DatanodeNetworkErrors
Errors/s
Total number of network errors
id4hdfsdatanode, host4hdfsdatanode
HdfsDnHbRtHeartbeatsavgtime
Heartbeat delay_HeartbeatsAvgTime
ms
Average heartbeat API time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnHbOpsHeartbeatsnumops
Heartbeat QPS_HeartbeatsNumOps
Heartbeats/s
Heartbeat API QPS
id4hdfsdatanode, host4hdfsdatanode
HdfsDnDatapacketAvgtimeSend datapackettransfer nanosavgtime
Packet transfer operations QPS_SendDataPacketTransfer NanosAvgTime
ms
Average data packet sending time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsRead blockopnumops
Data block operations _ReadBlockOpNumOps
Operations/s
OPS of block reads from the DataNode
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsWrite blockopnumops
Data block operations _WriteBlockOpNumOps
Operations/s
OPS of block writes to the DataNode
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsBlock checksumopnumops
Data block operations _BlockChecksumOpNumOps
Operations/s
OPS of Checksum operations by the DataNode
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsCopy blockopnumops
Data block operations _CopyBlockOpNumOps
Operations/s
OPS of block copying operations
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsReplace blockopnumops
Data block operations _ReplaceBlockOpNumOps
Operations/s
OPS of Replace Block operations
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsBlock reportsnumops
Data block operations _BlockReportsNumOps
Operations/s
OPS of block reporting operations
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsIncremental blockreports numops
Data block operations _IncrementalBlockReports NumOps
Operations/s
OPS of incremental block reporting
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsCache reportsnumops
Data block operations _CacheReportsNumOps
Operations/s
OPS of cache reporting
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksOpsPacketack roundtriptimenanos numops
Data block operations _PacketAckRoundTripTimeNanos NumOps
Operations/s
Number of ACK ROUND TRIP operations processed per second
id4hdfsdatanode, host4hdfsdatanode
HdfsDnFsyncOpsFsync nanosnumops
Fsync operations _FsyncNanosNumOps
Operations/s
Number of Fsync operations
id4hdfsdatanode, host4hdfsdatanode
HdfsDnFlushOpsFlush nanosnumops
Flush operations _FlushNanosNumOps
Operations/s
Number of Flush operations processed per second
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtRead blockopavgtime
Data block operation delay _ReadBlockOpAvgTime
ms
Average block read time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtWrite blockopavgtime
Data block operation delay _ReplaceBlockOpAvgTime
ms
Average block write time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtBlock checksumopavgtime
Data block operation delay_BlockChecksumOpAvgTime
ms
Average block check time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtCopy blockopavgtime
Data block operation delay _CopyBlockOpAvgTime
ms
Average block copy time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRt Replaceblockopavgtime
Data block operation delay _Replaceblockopavgtime
ms
Average Replace Block operation time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtBlock reportsavgtime
Data block operation delay _BlockReportsAvgTime
ms
Average block reporting time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtIncremental blockreportsavgtime
Data block operation delay _IncrementalBlockReportsAvgTime
ms
Average time of incremental block reporting
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtCache reportsavgtime
Data block operation delay _CacheReportsAvgTime
ms
Average time of cache reporting
id4hdfsdatanode, host4hdfsdatanode
HdfsDnBlocksRtPacketack roundtriptimenanos avgtime
Data block operation delay _PacketAckRoundTripTimeNanos AvgTime
ms
Average time of processing ACK ROUND TRIP
id4hdfsdatanode, host4hdfsdatanode
HdfsDnFlushRtFlushnanosavgtime
Flush delay_FlushNanosAvgTime
ms
Average Flush operation time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnFsyncRtFsyncnanosavgtime
Fsync delay_FsyncNanosAvgTime
ms
Average Fsync operation time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskblockswrite
RAMDISKBlocks_Ram DiskBlocksWrite
Blocks/s
Total number of blocks written to the memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskblockswritefallback
RAMDISKBlocks_Ram DiskBlocksWriteFallback
Blocks/s
Total number of blocks failed to be written to the memory (failover to the disk)
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOpRamdisk blocksdeletedbeforelazypersisted
RAMDISKBlocks_RamDiskBlocks DeletedBeforeLazyPersisted
Blocks/s
Total number of blocks deleted before the application is saved to the disk
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskblocksreadhits
RAMDISKBlocks_Ram DiskBlocksReadHits
Blocks/s
Total number of reads from the blocks in the memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskblocksevicted
RAMDISKBlocks_Ram DiskBlocksEvicted
Blocks/s
Total number of blocks cleared in the memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOpRamdisk blocksevictedwithoutread
RAMDISKBlocks_RamDiskBlocks EvictedWithoutRead
Blocks/s
Total number of blocks retrieved from the memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskblockslazypersisted
RAMDISKBlocks_RamDisk BlocksLazyPersisted
Blocks/s
Number of disk writes by the lazy writer
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksOp Ramdiskbyteslazypersisted
RAMDISKBlocks_Ram DiskBytesLazyPersisted
Bytes/s
Total number of bytes written to the disk by the lazy writer
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRamBlocksBytes Ramdiskbyteswrite
RAM Disk write speed_RamDiskBytesWrite
Bytes/s
Total number of bytes written to the memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMem Memnonheapusedm
JVM memory _MemNonHeapUsedM
MB
Size of the NonHeapMemory currently used by JVM
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMem Memnonheapcommittedm
JVM memory _MemNonHeapCommittedM
MB
Size of the NonHeapCommittedM configured by JVM
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMemMemheapusedm
JVM memory _MemHeapUsedM
MB
Size of the HeapMemory currently used by JVM
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMem Memheapcommittedm
JVM memory _MemHeapCommittedM
MB
Size of the HeapMemory committed by JVM
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMemMemheapmaxm
JVM memory _MemHeapMaxM
MB
Size of the HeapMemory configured by JVM
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmMemMemmaxm
JVM memory _MemMaxM
MB
Maximum memory size that can be used by JVM during runtime
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreadsThreadsnew
Number of JVM threads _ThreadsNew
Count
Number of threads in the new state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreads Threadsrunnable
Number of JVM threads _ThreadsRunnable
Count
Number of threads in the runnable state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreads Threadsblocked
Number of JVM threads _ThreadsBlocked
Count
Number of threads in the blocked state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreads Threadswaiting
Number of JVM threads _ThreadsWaiting
Count
Number of threads in the WAITING state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreads Threadstimedwaiting
Number of JVM threads _ThreadsTimedWaiting
Count
Number of threads in the TIMED WAITING state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmJavaThreads Threadsterminated
Number of JVM threads _ThreadsTerminated
Count
Number of threads in the Terminated state
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmLogTotalLogfatal
Number of JVM logs_LogFatal
Count
Number of Fatal logs
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmLogTotalLogerror
Number of JVM logs_LogError
Count
Number of Error logs
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmLogTotalLogwarn
Number of JVM logs_LogWarn
Count
Number of Warn logs
id4hdfsdatanode, host4hdfsdatanode
HdfsDnJvmLogTotalLoginfo
Number of JVM logs_LogInfo
Count
Number of Info logs
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryS0
Memory space percentage_S0
%
Percentage of used Survivor 0 memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryS1
Memory space percentage_S1
%
Percentage of used Survivor 1 memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryE
Memory space percentage_E
%
Percentage of used Eden memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryO
Memory space percentage_O
%
Percentage of used Old memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryM
Memory space percentage_M
%
Percentage of used Metaspace memory
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilMemoryCcs
Memory space percentage_CCS
%
Percentage of memory used by compressed class space
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilGcCountFgc
GC count_FGC
Count
Full GC count
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilGcCountYgc
GC count_YGC
Count
Young GC count
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilGcTimeYgct
GC time_YGCT
s
Time consumed by Young GC
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilGcTimeFgct
GC time_FGCT
s
Time consumed by Full GC
id4hdfsdatanode, host4hdfsdatanode
HdfsDnGcUtilGcTimeGct
GC time_GCT
s
Time used to collect garbage
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004RxtxReceivedbytes
Data traffic_ReceivedBytes
Bytes/s
Data receiving rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004RxtxSentbytes
Data traffic_SentBytes
Bytes/s
Data sending rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004QpsRpc queuetimenumops
QPS_RpcQueueTimeNumOps
Queries/s
RPC call rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004RtRpc queuetimeavgtime
Request processing delay _RpcQueueTimeAvgTime
ms
Average RPC delay
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004AuthRpc authenticationfailures
Authentication and authorization _RpcAuthenticationFailures
Failures/s
Number of RPC authentication failures
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004AuthRpc authenticationsuccesses
Authentication and authorization _RpcAuthenticationSuccesses
Successes/s
Number of RPC authentication successes
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004AuthRpc authorizationfailures
Authentication and authorization _RpcAuthorizationFailures
Failures/s
Number of RPC authorization failures
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004AuthRpc authorizationsuccesses
Authentication and authorization _RpcAuthorizationSuccesses
Successes/s
Number RPC authorization successes
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004Connections Numopenconnections
Number of current connections _NumOpenConnections
Count
Number of current connections
id4hdfsdatanode, host4hdfsdatanode
HdfsDnPort4004QueueLen Callqueuelength
RPC processing queue length _CallQueueLength
Count
Length of the current RPC processing queue
id4hdfsdatanode, host4hdfsdatanode
HdfsDnThreadTimeCurrent threadcputime
CPU time _CurrentThreadCpuTime
ms
CPU time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnThreadTimeCurrent threadusertime
CPU time _CurrentThreadUserTime
ms
User time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnStartTimeStarttime
Start time_StartTime
s
Process start time
id4hdfsdatanode, host4hdfsdatanode
HdfsDnThreadCount Peakthreadcount
Number of threads_PeakThreadCount
Count
Peak number of threads
id4hdfsdatanode, host4hdfsdatanode
HdfsDnThreadCount Daemonthreadcount
Number of threads_DaemonThreadCount
Count
Number of background threads
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRtWrit
Read/write delay_Write
MB/s
Disk write rate
id4hdfsdatanode, host4hdfsdatanode
HdfsDnRtRead
Read/write delay_Read
Queries/s
Read QPS
id4hdfsdatanode, host4hdfsdatanode
HdfsDnDatapacketOps Datapacketops
Packet transfer QPS_DataPacketOps
Queries/s
Packet transfer QPS
id4hdfsdatanode, host4hdfsdatanode

HDFS - Journal Node

Parameter
Metric Name
Unit
Description
Dimension
HdfsJnJvmMemMemnon
heapusedm
JVM memory_MemNonHeapUsedM
MB
Size of the NonHeapMemory currently used by JVM
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmMemMemnon
heapcommittedm
JVM memory_MemNonHeapCommittedM
MB
JVM memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmMemMem
heapusedm
JVM memory_MemHeapUsedM
MB
Size of the HeapMemory currently used by JVM
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmMemMem
heapcommittedm
JVM memory_MemHeapCommittedM
MB
Size of the HeapMemory committed by JVM
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmMemMem
heapmaxm
JVM memory_MemHeapMaxM
MB
Size of the HeapMemory configured by JVM
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmMemMemmaxm
JVM memory_MemMaxM
MB
Maximum memory size that can be used by JVM during runtime
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadsnew
Number of JVM threads_ThreadsNew
Count
Number of threads in the new state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadsrunnable
Number of JVM threads_ThreadsRunnable
Count
Number of threads in the runnable state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadsblocked
Number of JVM threads_ThreadsBlocked
Count
Number of threads in the blocked state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadswaiting
Number of JVM threads_ThreadsWaiting
Count
Number of threads in the WAITING state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadstimedwaiting
Number of JVM threads_ThreadsTimedWaiting
Count
Number of threads in the TIMED WAITING state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmJavaThreads
Threadsterminated
Number of JVM threads_ThreadsTerminated
Count
Number of threads in the Terminated state
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmLogTotalLogfatal
Number of JVM logs_LogFatal
Count
Number of Fatal logs
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmLogTotalLogerror
Number of JVM logs_LogError
Count
Number of Error logs
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmLogTotalLogwarn
Number of JVM logs_LogWarn
Count
Number of Warn logs
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnJvmLogTotalLoginfo
Number of JVM logs_LogInfo
Count
Number of Info logs
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryS0
Memory space percentage_S0
%
Percentage of used Survivor 0 memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryS1
Memory space percentage_S1
%
Percentage of used Survivor 1 memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryE
Memory space percentage_E
%
Percentage of used Eden memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryO
Memory space percentage_O
%
Percentage of used Old memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryM
Memory space percentage_M
%
Percentage of used Metaspace memory
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilMemoryCcs
Memory space percentage_CCS
%
Percentage of memory used by compressed class space
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilGcCountFgc
GC count_FGC
Count
Full GC count
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilGcCountYgc
GC count_YGC
Count
Young GC count
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilGcTimeYgct
GC time_YGCT
s
Time consumed by Young GC
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilGcTimeFgct
GC time_FGCT
s
Time consumed by Full GC
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnGcUtilGcTimeGct
GC time_GC
s
Time used to collect garbage
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005Rxtx
Receivedbytes
Data traffic_ReceivedBytes
Bytes/s
Data receiving rate
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005Rxtx
Receivedbytes
Data traffic_ReceivedBytes
Bytes/s
Data sending rate
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005QpsRpc
queuetimenumops
QPS_RpcQueueTimeNumOps
Queries/s
RPC call rate
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005RtRpc
queuetimeavgtime
Request processing delay_RpcQueueTimeAvgTime
ms
Average RPC delay
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005AuthRpc
authenticationfailures
Authentication and authorization_RpcAuthenticationFailures
Failures/s
Number of RPC authentication failures
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005AuthRpc
authorizationsuccesses
Authentication and authorization_RpcAuthorizationSuccesses
Successes/s
Number of RPC authorization successes
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005AuthRpc
authenticationsuccesses
Authentication and authorization_RpcAuthenticationSuccesses
Successes/s
Number of RPC authentication successes
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005AuthRpc
authorizationfailures
Authentication and authorization_RpcAuthorizationFailures
Failures/s
Number of RPC authorization failures
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005Connections
Numopenconnections
Number of current connections_NumOpenConnections
Count
Number of current connections
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnPort4005QueueLen
Callqueuelength
RPC processing queue length_CallQueueLength
Count
Length of the current RPC processing queue
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnThreadTimeCurrent
threadcputime
CPU time_CurrentThreadCpuTime
ms
CPU time
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnThreadTimeCurrent
threadusertime
CPU time_CurrentThreadUserTime
ms
User time
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnStartTimeStarttime
Start time_StartTime
s
Process start time
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnThreadCount
Threadcount
Number of threads_ThreadCount
Count
Number of threads
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnThreadCount
Peakthreadcount
Number of threads_PeakThreadCount
Count
Peak number of threads
host4hdfsjournalnode,
id4hdfsjournalnode
HdfsJnThreadCountDaemon
threadcount
Number of threads_DaemonThreadCount
Count
Number of background threads
host4hdfsjournalnode,
id4hdfsjournalnode

HDFS - ZKFC

Parameter
Metric Name
Unit
Description
Dimension
HdfsDfzkGcUtilMemoryS0
Memory space percentage_S0
%
Percentage of used Survivor 0 memory
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilMemoryS1
Memory space percentage_S1
%
Percentage of used Survivor 1 memory
host4hdfszkfailovercontroller, id4hdfszkfailovercontroller
HdfsDfzkGcUtilMemoryE
Memory space percentage_E
%
Percentage of used Eden memory
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilMemoryO
Memory space percentage_O
%
Percentage of used Old memory
host4hdfszkfailovercontroller, id4hdfszkfailovercontroller
HdfsDfzkGcUtilMemoryM
Memory space percentage_M
%
Percentage of used Metaspace memory
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilMemoryCcs
Memory space percentage_CCS
%
Percentage of memory used by compressed class space
host4hdfszkfailovercontroller, id4hdfszkfailovercontroller
HdfsDfzkGcUtilGcCountFgc
GC count_FGC
Count
Full GC count
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilGcCountYgc
GC count_YGC
Count
Young GC count
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilGcTimeYgct
GC time_YGCT
s
Time consumed by Young GC
host4hdfszkfailovercontroller, id4hdfszkfailovercontroller
HdfsDfzkGcUtilGcTimeFgct
GC time_FGCT
s
Time consumed by Full GC
host4hdfszkfailovercontroller,
id4hdfszkfailovercontroller
HdfsDfzkGcUtilGcTimeGct
GC time_GCT
s
Time used to collect garbage
host4hdfszkfailovercontroller, id4hdfszkfailovercontroller
Note:
The statistical granularity (period) may vary by metric. The DescribeBaseMetrics API can be used to obtain the period supported by each metric.

Overview of the Parameters in Each Dimension

Parameter Name
Dimension Name
Dimension Description
Format
Instances.N.Dimensions.0.Name
id4hdfsoverview
Dimension name of the EMR instance ID
String-type dimension name, such as id4hdfsoverview
Instances.N.Dimensions.0.Value
id4hdfsoverview
Specific EMR instance ID
Specific EMR instance ID, such as emr-mm8bs222
Instances.N.Dimensions.1.Name
host4hdfsoverview
Dimension name of the node IP in the EMR instance
String-type dimension name, such as host4hdfsoverview
Instances.N.Dimensions.1.Value
host4hdfsoverview
Specific node IP in the EMR instance
Specific node IP, such as 1.1.1.1
Instances.N.Dimensions.0.Name
id4hdfsnamenode
Dimension name of the EMR instance ID
String-type dimension name, such as id4hdfsnamenode
Instances.N.Dimensions.0.Value
id4hdfsnamenode
Specific EMR instance ID
Specific EMR instance ID, such as emr-mm8bs222
Instances.N.Dimensions.1.Name
host4hdfsnamenode
Dimension name of the node IP in the EMR instance
String-type dimension name, such as host4hdfsnamenode
Instances.N.Dimensions.1.Value
host4hdfsnamenode
Specific node IP in the EMR instance
Specific node IP, such as 1.1.1.1
Instances.N.Dimensions.0.Name
id4hdfsdatanode
Dimension name of the EMR instance ID
String-type dimension name, such as id4hdfsdatanode
Instances.N.Dimensions.0.Value
id4hdfsdatanode
Specific EMR instance ID
Specific EMR instance ID, such as emr-mm8bs222
Instances.N.Dimensions.1.Name
host4hdfsdatanode
Dimension name of the node IP in the EMR instance
String-type dimension name, such as host4hdfsdatanode
Instances.N.Dimensions.1.Value
host4hdfsdatanode
Specific node IP in the EMR instance
Specific node IP, such as 1.1.1.1
Instances.N.Dimensions.0.Name
id4hdfsjournalnode
Dimension name of the EMR instance ID
String-type dimension name, such as id4hdfsjournalnode
Instances.N.Dimensions.0.Value
id4hdfsjournalnode
Specific EMR instance ID
Specific EMR instance ID, such as emr-mm8bs222
Instances.N.Dimensions.1.Name
host4hdfsjournalnode
Dimension name of the node IP in the EMR instance
String-type dimension name, such as host4hdfsjournalnode
Instances.N.Dimensions.1.Value
host4hdfsjournalnode
Specific node IP in the EMR instance
Specific node IP, such as 1.1.1.1
Instances.N.Dimensions.0.Name
id4hdfszkfailovercontroller
Dimension name of the EMR instance ID
String-type dimension name, such as id4hdfszkfailovercontroller
Instances.N.Dimensions.0.Value
id4hdfszkfailovercontroller
Specific EMR instance ID
Specific EMR instance ID, such as emr-mm8bs222
Instances.N.Dimensions.1.Name
host4hdfszkfailovercontroller
Dimension name of the node IP in the EMR instance
String-type dimension name, such as host4hdfszkfailovercontroller
Instances.N.Dimensions.1.Value
host4hdfszkfailovercontroller
Specific node IP in the EMR instance
Specific node IP, such as 1.1.1.1

Input Parameters

EMR (HDFS) supports querying monitoring data based on the following six combinations of dimensions. The values for the input parameters are as follows:
1. To query the metric monitoring data of HDFS - OverviewAggregation, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfsoverview &Instances.N.Dimensions.0.Value=EMR instance ID
2. To query the metric monitoring data of HDFS - Overview, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfsoverview &Instances.N.Dimensions.0.Value=EMR instance ID &Instances.N.Dimensions.1.Name=host4hdfsoverview &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
3. To query the metric monitoring data of HDFS - NameNode, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfsnamenode &Instances.N.Dimensions.0.Value=EMR instance ID &Instances.N.Dimensions.1.Name=host4hdfsnamenode &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
4. To query the metric monitoring data of HDFS - DataNode, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfsdatanode &Instances.N.Dimensions.0.Value=EMR instance ID &Instances.N.Dimensions.1.Name=host4hdfsdatanode &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
5. To query the metric monitoring data of HDFS - Journal Node, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfsjournalnode &Instances.N.Dimensions.0.Value=Specific EMR instance ID &Instances.N.Dimensions.1.Name=host4hdfsjournalnode &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
6. To query the metric monitoring data of HDFS - ZKFC, use the following input parameters: &Namespace=QCE/TXMR_HDFS &Instances.N.Dimensions.0.Name=id4hdfszkfailovercontroller &Instances.N.Dimensions.0.Value=Specific EMR instance ID &Instances.N.Dimensions.1.Name=host4hdfszkfailovercontroller &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon