TDMQ for Apache Pulsar
- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Pricing Overview
- Pro Cluster
- Getting Started
- Using SDK to Send and Receive Normal Messages
- Developer Guide
- How It Works
- Operation Guide
- Cluster Management
- Monitoring and Alarms
- Permission Management
- Pulsar Instance Permission Management
- Cross-Region Replication
- Practical Tutorial
- API Documentation
- Making API Requests
- Cluster APIs
- Namespace APIs
- CMQ Management APIs
- Environment Role APIs
- TDMQ for RabbitMQ APIs
- RocketMQ APIs
- TDMQ for RocketMQ APIs
- Production and Consumption APIs
- SDK Documentation
- Apache Pulsar TCP Protocol
- General References
Connecting to Prometheus
Last updated: 2025-04-01 11:37:52
TDMQ for Apache Pulsar supports integrating pro cluster monitoring data into users' self-built Prometheus, which facilitates the observation of the TDMQ cluster's operations. It also allows for timely HPA of business workloads through Prometheus monitoring data, making overall online ops more automated.
Advanced Usage:
Using the standard Prometheus monitoring format, you can access data with your own Prometheus through our provided Exporter;
For instance, configuring a Prometheus data source in Grafana or accessing Prometheus data with K8s for Workload HPA, etc.
Note：
If your professional cluster was created before March 25, 2024, enabling this feature requires a cluster upgrade. Contact us for assistance.
User Guide
1. Log in to the TDMQ for Apache Pulsar Pro Cluster console.
2. In the left sidebar, choose Cluster Management, and click the target instance's ID to enter the instance basic information page.
3. Click to enter the Monitoring Information Sheet page.
4. In the top right corner, click the TMP Entry Access button to Access Monitoring Targets, and select the appropriate network type and network configuration.
5. Click Submit to access a set of monitoring targets.
6. Modify the configuration file prometheus.yml to add a node_exporter scraping task.
job_name: prometheus-tdmq-pulsarhonor_timestamps: truescrape_interval: 60smetrics_path: /tencent-cloud-metrics/scheme: httpstatic_configs:- targets:- 10.x.x.x:10001- 10.x.x.x:10002
When
honor_timestamps is set to true, Prometheus will use the timestamps provided by the Exporter for its metrics rather than using the timestamps at which the Prometheus server receives these metrics.scrape_interval, the frequency of scraping monitoring metrics data. Currently at minute-level.metrics_path, the path to access monitoring metrics. Set it to /tencent-cloud-metrics/.scheme, the protocol for accessing resources in the configuration. Currently it only supports http.7. Access the corresponding visualization interface to view the configured monitoring metrics.
Note:
1. This feature provides monitoring data at the cluster level, Topic level, and subscription level for pro clusters, aligning perfectly with the metric items on the console. However, unlike the console, where the monitoring data is pre-aggregated, the data through Prometheus is post-aggregated and requires you to manually aggregate and display it on the Grafana dashboard.
2. Due to the different paths of monitoring data collection and different aggregation logic, the data captured by this feature may differ in specific values from the data displayed on the user console.
3. This feature is only supported by pro clusters.
Metric Description
Cluster Level
Metric | Metric Name |
Cluster message production rate | pulsar_caculate_rate_in |
Cluster message consumption rate | pulsar_caculate_rate_out |
Cluster message storage size | pulsar_storage_size |
Cluster production bandwidth peak | pulsar_throughput_in |
Cluster consumption bandwidth peak | pulsar_throughput_out |
Topic Level
Metric | Metric Name |
Message production rate | pulsar_caculate_rate_in |
Message consumption rate | pulsar_caculate_rate_out |
Message production throughput | pulsar_throughput_in |
Message consumption throughput | pulsar_throughput_out |
Producer count | pulsar_producers_count |
Consumer count | pulsar_consumers_count |
Message storage size | pulsar_storage_size |
Message backlog count | pulsar_msg_backlog |
Message backlog size | pulsar_storage_backlog_size |
Producer rate limiting count | pulsar_publish_rate_limit_times |
Filtered message count | pulsar_tag_filter_rejected_msg_rate |
Subscription Level
Metric | Metric Name |
Message backlog entries | pulsar_subscription_back_log |
Message consumption rate | pulsar_subscription_msg_rate_out |
Message consumption throughput | pulsar_subscription_msg_throughput_out |
Unacknowledged message count | pulsar_subscription_unacked_messages |
Consumer count | pulsar_subscription_consumers_count |
Delayed message count | pulsar_subscription_delayed |
Message deletion rate | pulsar_subscription_msg_rate_expired |
Filtered message count | pulsar_subscription_tag_filter_rejected_msg_rate |
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No