Monitoring and Alarm Configuration

Overview
Cloud Service for etcd provides monitoring data of node resource metrics, application metrics, instance-level metrics, and instance API metrics by default. Alarm rules can be set for these metrics. In addition, if you want to display custom monitoring metrics, you can use TMP as an additional monitoring service and configure custom monitoring metrics.
Prerequisites
etcd clusters have been created.
Directions
1. Log in to the console of Cloud Service for etcd.
2. Go to the etcd cluster list page, select the region at the top of the page, and find the target etcd cluster in the cluster list below.
Viewing Monitoring Metrics
Click the 
﻿
 icon of the corresponding cluster in the cluster list to enter the details page. Then, go to the Instance monitoring page.
Metric Description
Aggregation method:
1. Time range: Time range of the data displayed in monitoring charts.
2. Granularity: Aggregation method for each data point in charts. Raw data is collected every 15 seconds by default. The console supports displaying the average value of data aggregated by 1 minute or 5 minutes.
3. Automatic refresh: Interval for automatically refreshing the charts, which can be 30 seconds, 5 minutes, 30 minutes, or 1 hour. This feature is disabled by default.
Metric overview:
Metric Dimension
Metric Name
Unit
Metric Description
Node resource
CPU usage percentage
Percentage
Proportion of the current node CPU usage to Pod CPU request value of the current node (the CPU specification selected during instance creation).
Node resource
File system read rate
MiBytes/s
Volume of data read from the node data disk per second.
Node resource
File system write rate
MiBytes/s
Volume of data written to the node data disk per second.
Node resource
Memory usage percentage
Percentage
Proportion of the current node memory usage to Pod MEM request value of the current node (the memory specification selected during instance creation).
Node resource
Total memory usage
MiBytes
Memory usage of the node.
Node resource
Network receiving rate
MiBytes
Data receiving rate of the node NIC.
Node resource
Network sending rate
MiBytes
Data sending rate of the node NIC.
Application metric
Number of database keys
Count
Number of keys on the node. The data is obtained from etcd metrics. Calculation formula: etcd_debugging_mvcc_keys_total{job="$job"}.
Application metric
Database MVCC writes
Times
Number of data writes on the node. Calculation formula: etcd_mvcc_put_total{job="$job"}.
Application metric
Database size
MiBytes
Database size calculated on the node. Calculation formula: etcd_debugging_mvcc_db_total_size_in_bytes{job="$job"}.
Application metric
Consensus proposal application rate
Time/s
The value is usually small. (Only a few thousand proposals are applied even under high load). If the value keeps increasing, it indicates that the etcd server is overloaded, which may be caused by queries with high consumption (such as large-range queries or large txn operations). Calculation formula: rate(etcd_server_proposals_applied_total{job="$job"}[5m]).
Application metric
Consensus proposal commit rate
Time/s
The value usually increases over time. If the delay between a single member and the leader is high for a long time, it indicates that the member is running slowly or is not healthy. Calculation formula: rate(etcd_server_proposals_committed_total{job="$job"}[5m]).
Application metric
Total number of queueing consensus proposals
Count
An increase of the value indicates that the client load is high or the member cannot submit proposals. Calculation formula: etcd_server_proposals_pending{job="$job"}.
Application metric
Growth rate of failed consensus proposals
Time/s
This metric is usually related to two issues: temporary failures related to leader election or longer-term failures due to the lack of the arbitration node in the cluster. Calculation formula: rate(etcd_server_proposals_failed_total{job="$job"}[5m]).
Instance-level metric
Cluster leader existence
Boolean value
If there is no leader, the instance is unavailable. Calculation formula: max(etcd_server_has_leader{job="$job"}).
Instance-level metric
Total number of leader switches
Times
If there is no leader, frequent leader changes will great affect the etcd performance, which may be caused by network connection issues or high load on the etcd cluster. Calculation formula: max(etcd_server_leader_changes_seen_total{job="$job"}).
Note:
The data collected for this metric is the aggregated data after the cluster is successfully created. It is irrelevant to the alarm cycle.
Instance API
gRPC call rate
Time/s
gRPC call rate of operations using the specific method. Calculation formula: sum(rate(grpc_server_handled_total{job="$job"}[1m])) by (job,grpc_method,instance).
﻿

Was this page helpful?

You can also Contact Sales or Submit a Ticket for help.

Yes

Metric Dimension	Metric Name	Unit	Metric Description
Node resource	CPU usage percentage	Percentage	Proportion of the current node CPU usage to Pod CPU request value of the current node (the CPU specification selected during instance creation).
Node resource	File system read rate	MiBytes/s	Volume of data read from the node data disk per second.
Node resource	File system write rate	MiBytes/s	Volume of data written to the node data disk per second.
Node resource	Memory usage percentage	Percentage	Proportion of the current node memory usage to Pod MEM request value of the current node (the memory specification selected during instance creation).
Node resource	Total memory usage	MiBytes	Memory usage of the node.
Node resource	Network receiving rate	MiBytes	Data receiving rate of the node NIC.
Node resource	Network sending rate	MiBytes	Data sending rate of the node NIC.
Application metric	Number of database keys	Count	Number of keys on the node. The data is obtained from etcd metrics. Calculation formula: etcd_debugging_mvcc_keys_total{job="$job"}.
Application metric	Database MVCC writes	Times	Number of data writes on the node. Calculation formula: etcd_mvcc_put_total{job="$job"}.
Application metric	Database size	MiBytes	Database size calculated on the node. Calculation formula: etcd_debugging_mvcc_db_total_size_in_bytes{job="$job"}.
Application metric	Consensus proposal application rate	Time/s	The value is usually small. (Only a few thousand proposals are applied even under high load). If the value keeps increasing, it indicates that the etcd server is overloaded, which may be caused by queries with high consumption (such as large-range queries or large txn operations). Calculation formula: rate(etcd_server_proposals_applied_total{job="$job"}[5m]).
Application metric	Consensus proposal commit rate	Time/s	The value usually increases over time. If the delay between a single member and the leader is high for a long time, it indicates that the member is running slowly or is not healthy. Calculation formula: rate(etcd_server_proposals_committed_total{job="$job"}[5m]).
Application metric	Total number of queueing consensus proposals	Count	An increase of the value indicates that the client load is high or the member cannot submit proposals. Calculation formula: etcd_server_proposals_pending{job="$job"}.
Application metric	Growth rate of failed consensus proposals	Time/s	This metric is usually related to two issues: temporary failures related to leader election or longer-term failures due to the lack of the arbitration node in the cluster. Calculation formula: rate(etcd_server_proposals_failed_total{job="$job"}[5m]).
Instance-level metric	Cluster leader existence	Boolean value	If there is no leader, the instance is unavailable. Calculation formula: max(etcd_server_has_leader{job="$job"}).
Instance-level metric	Total number of leader switches	Times	If there is no leader, frequent leader changes will great affect the etcd performance, which may be caused by network connection issues or high load on the etcd cluster. Calculation formula: max(etcd_server_leader_changes_seen_total{job="$job"}). Note: The data collected for this metric is the aggregated data after the cluster is successfully created. It is irrelevant to the alarm cycle.
Instance API	gRPC call rate	Time/s	gRPC call rate of operations using the specific method. Calculation formula: sum(rate(grpc_server_handled_total{job="$job"}[1m])) by (job,grpc_method,instance).

tencent cloud

New User Offers

Next-Generation CDN：EdgeOne

Elasticsearch Service Special Offers

Free Tier

Tencent Cloud Startup Program

Special Offers

Lighthouse Special Offers

Cloud Object Storage Special Offers

Featured Products

New Products

Education

Tencent Cloud Online Education Solutions

Gaming

Gaming Solution

Game Media Solutions

Financial Services

Financial Services Solution

Audio & Video

Audio/Video Solution

LVB Recording Solution

Interactive Classroom Solution

Interactive Live Streaming Solution

Audio Chat Social Networking Solution

Real Estate

Tencent Cloud LinkBase(Weiling)

E-commerce

E-commerce retail solutions

Compute

Cloud Virtual Machine

Auto Scaling

Batch Compute

CVM Dedicated Host

Database

TencentDB for MySQL

TencentDB for Redis®

TencentDB for CTSDB

TDSQL for MySQL

Data Transfer Service

TencentDB for MongoDB

TencentDB for PostgreSQL

TencentDB for SQL Server

TencentDB for TcaplusDB

Video Service

Cloud Streaming Services

Video on Demand

Media Processing Service

Cloud Application Rendering

Cloud Contact Center

Game Multimedia Engine

Chat

Real-time Communication

Tencent Effect SDK

AI and Machine Learning

Image Creation Large Model

Face Fusion

eKYC

Optical Character Recognition

Video Creation Large Model

Industry Applications

Tencent HealthCare Omics Platform

Container and Middleware

TDMQ for CKafka

Serverless Cloud Function

Tencent Kubernetes Engine

Tencent Kubernetes Engine for Serverless

Networking

Cloud Load Balancer

Virtual Private Cloud

Direct Connect

Cloud Connect Network

NAT Gateway

VPN Connection

Bandwidth Package

Anycast Internet Acceleration

Elastic Network Interface

Flow Logs

Global Application Acceleration Platform

Security

Captcha