tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Cloud Observability Platform
    Documentation
    Download PDF

    Alarm Suppression

    Last updated: 2024-08-22 16:20:00
    Download PDF

      Foreword

      To avoid additional Ops workload caused by hundreds of similar alarms due to the same issue, we have introduced the alarm suppression feature. Alarm suppression means that if an alarm of a certain type is triggered, other related similar alarms will be suppressed. For example, if the alarm content is that a certain cluster is inaccessible, you can configure Inhibition rules to silence all other alarms related to that cluster.

      Directions

      1. Log in to TMP Console.
      2. In the Prometheus instance list, click Instance ID/Name.
      3. Enter the Prometheus Management Center, and click Alarm Management > Inhibit Rules > Create in the top navigation bar.
      
      
      
      4. After navigating to the Create page, configure the suppression rules as prompted by the page, then click Save.
      
      
      

      Parameter Description

      Parameter
      Description
      Source Matcher
      Triggered alarm. Select Label name, Condition, and Label value.
      Target Matcher
      Alarms to be silenced. Select Label name, Condition, and Label value.
      Equal
      The target and source alarm must have the same label value for the label name in the matching criteria. Select Label name.
      Note:
      Inhibition rules configuration: When there is an alarm (source) that meets a certain rule, the suppression rule will silence another alarm (target) that meets a different rule. The target and source alarm must have the same label value for the label name in the matching criteria.
      To prevent self-suppression alarms, alarms that match both the target and source rules cannot be suppressed by other alarms that also match both target and source rules (including themselves). Therefore, it is recommended to design the source and target rules of alarms in such a way that no alarm matches both the source and target rules simultaneously.

      Example

      Use Cases: Alarm on High Server CPU Load

      Scenario Description:

      In a monitoring system, two alarms are configured:
      Alarm A: CPU load exceeds 90%.
      Alarm B: System response time exceeds 500 ms.
      Both alarms are triggered by the same cause: high CPU load on the server, leading to degraded system performance. The policy rules for Alarm A are as follows: alert: HighCPUUsage expr: avg(rate(cpu_usage_seconds_total[5m])) by (instance) > 0.9 The policy rules for Alarm B are as follows: alert: HighResponseTime expr: avg(response_time_seconds) by (instance) > 0.5 The Inhibition rule configuration is as follows:
      Source: alert=HighCPUUsage
      Target: alert=HighResponseTime
      Matching criteria: instance

      Overall Effect:

      The average rate of the cpu_usage_seconds_total metric over 5 minutes is 95%. If the metric's label instance=instanceX, Alarm A will be triggered, and an alarm notification will be sent.
      The average value of the response_time_seconds metric is 0.8s. If the metric's label instance=instanceX, Alarm B will be triggered, but no alarm notification will be sent because the Inhibition rule is matched.
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy