tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Cloud Log Service
    Documentation
    Download PDF

    Metrics Reporting

    Last updated: 2024-09-20 17:48:27
    Download PDF
      Metric topics support the Prometheus Remote Write protocol, allowing various collectors compatible with this protocol to collect and report metrics to the metrics topics, such as vmagent and telegraf. With these collectors, you can report various types of monitoring metrics to the metric topics, such as:
      Server monitoring metrics, such as CPU, memory, and disk utilization.
      Container cluster monitoring metrics, such as Pods memory usage and apiserver request delay.
      Middleware monitoring metrics, such as performance metrics for MySQL, Kafka, Nginx, and other middleware.
      Application system monitoring metrics, such as API throughput, response time, and the number of error requests.
      Metric topics can also serve as remote storage for a local Prometheus instance. Using the Remote Write method, metrics from the local Prometheus can be reported to the metric topics.

      Reporting Address

      Network Type
      address
      Public Network
      https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write
      Private Network
      https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
      Among them:
      Replace ${region} with the region where the metric topic is located, such as ap-beijing. For more region abbreviations, see Available Regions. Currently, only the regions of Beijing, Shanghai, Guangzhou, and Nanjing are supported.
      Replace ${topicId} with the metric topic ID, such as 0e69453c-0727-4c9c-xxxx-ea51b10d2aba. You can find the topic ID in the Metric Topic List.

      Authentication method

      Metric reporting uses Basic Auth for authentication. Use the SecretId and SecretKey from the API Key as the username and password, respectively.
      username:${SecretId}
      password:${SecretKey}
      It is recommended to create a separate sub-account for the collector and use the SecretId and SecretKey of that account. Grant this sub-account only the following permissions to ensure security. For configuration details, see Sub-account Authorization.
      {
          "version": "2.0",
          "statement": [
              {
                  "effect": "allow",
                  "action": [
                      "cls:MetricsRemoteWrite"
                  ],
                  "resource": [
                      "*"
                  ]
              }
          ]
      }

      Configuration Example

      Telegraf

      Add the following output configuration to the Telegraf configuration file:
      [[outputs.http]]
        ## Reporting address: Replace ${region} and ${topicId}. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
        ## Private network address URL = https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
        url = "https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write"
      
        ## Authentication information: Replace ${SecretId} and ${SecretKey}.
        username = "${SecretId}"
        password = "${SecretKey}"
      
        ## Do not modify the Telegraf output data format configuration.
        data_format = "prometheusremotewrite"
        [outputs.http.headers]
           Content-Type = "application/x-protobuf"
           Content-Encoding = "snappy"
           X-Prometheus-Remote-Write-Version = "0.1.0"

      Vmagent

      When starting Vmagent, use the following parameters to configure remoteWrite:
      ./vmagent-prod \\
        -remoteWrite.url=https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write \\
        -remoteWrite.basicAuth.username=${SecretId} \\
        -remoteWrite.basicAuth.password=${SecretKey}
      Note:
      The remoteWrite.url is set to the public network address; if network conditions allow, it is recommended to use the private network address.
      The private network address is https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write.

      Prometheus

      Add the following configuration to the Prometheus configuration file:
      # Reporting address: Replace ${region} and ${topicId}. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
      # Private network address URL: https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
      url: https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write
      
      # Authentication information: Replace ${SecretId} and ${SecretKey}.
      basic_auth:
        username: ${SecretId}
        password: ${SecretKey}
      
      # Data write policy: Including caching and retry mechanisms, the following configuration is recommended for handling large data volumes.
      queue_config:
        capacity: 20480
        min_shards: 100
        max_samples_per_send: 2048
        batch_send_deadline: 20s
        min_backoff: 100ms
        max_backoff: 5s
      For more configuration details, see the official Prometheus documentation.

      Prometheus Operator

      Prometheus deployed in Kubernetes using the Prometheus Operator can report metrics in the following way:
      1. Use kubectl to create the Secret required for authentication, for example, with the following command:
      kubectl create secret generic kubepromsecret \\
       --from-literal=username=${SecretId} \\
       --from-literal=password=${SecretKey} \\
       -n monitoring
      Note:
      Replace ${SecretId} and ${SecretKey} in the command.
      If Prometheus is deployed in another namespace, modify -n monitoring to the correct namespace.
      You can also create Secret using a manifest file. For more details, see the official Kubernetes documentation.
      2. Open the Prometheus manifest configuration file, which is typically located in the GitHub repo file used by the Prometheus Operator. The usual path is kube-prometheus/manifests/prometheus-prometheus.yaml.
      3. Modify the Prometheus manifest configuration file by adding the following configuration at the end:
        remoteWrite:
        - url: "https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write"
          basicAuth:
            username:
              name: kubepromsecret
              key: username
            password:
              name: kubepromsecret
              key: password
      Note:
      Replace ${region} and ${topicId} in the configuration. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
      Private network address URL: https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
      When handling large data volumes, you can add the following data write policy, including caching and retry mechanisms. For more details, see the official Prometheus Operator documentation.
          queueConfig:
            capacity: 204800
            minShards: 100
            maxShards: 2048
            maxSamplesPerSend: 4096
            batchSendDeadline: 30s
            minBackoff: 100ms
            maxBackoff: 5s
      4. Apply the configuration file, for example, with the following command:
      kubectl apply -f prometheus-prometheus.yaml -n monitoring
      Note:
      Replace prometheus-prometheus.yaml with the correct configuration file path.
      If Prometheus is deployed in another namespace, modify -n monitoring to the correct namespace.
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy