- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
Background
When users execute fault actions via CFG, some unforeseen impacts may occur. To ensure these impacts remain within a controllable range, the platform provides guardrails to help users minimize losses and protect normal business operations.
Note:
Guardrails can be triggered and recovered by users through the Guardrail TencentCloud API. For details, see Trigger/Restore Guardrails Cloud API Interface Documentation. Current guardrails support two policies:
Pause the Experiment: When the guardrail is triggered by the user, the experiment will automatically pause.
Sequential Rollback of Actions: When the guardrail is triggered by the user, the pending recovery actions will be automatically executed. Once the necessary recovery actions are completed, the experiment will automatically pause.
If the guardrail is in the Trigger status and you manually continue to execute fault actions, the above policies will still be applied.
We do not recommend injecting faults into the same instance simultaneously. Otherwise, automatic recovery may fail, and users will need to manually execute the corresponding recovery actions.
Guardrail Use Method
Select Guardrail Policy
In Step 4 Global Configuration of Create an Experiment, you can select the guardrail policy for the current experiment, such as the Sequential Rollback of Actions policy. You can click How to trigger the guardrail? to view trigger code examples.
Guardrail Trigger Method
When selecting a guardrail policy, click How to trigger the guardrail? to view example code for calling the Trigger/Recover Guardrail TencentCloud API.
Execute an Experiment and Trigger Guardrails
After the experiment is created, execute the shutdown experiment. During the shutdown process, actively trigger the guardrail by calling the Guardrail Trigger TencentCloud API.
1. After triggering the guardrail by calling the TencentCloud API, the system will automatically execute the startup recovery action. Once completed, the experiment will be automatically paused.
2. You can click the Guardrail Policy tag to view the guardrail trigger records. In the guardrail trigger event, you can click Guardrail Details to view the guardrail's name and specific details.
3. You can click the Experiment Log button in the upper right corner to view the experiment logs, where you can see the guardrail execution logic logs.
4. Call the TencentCloud API to recover the guardrail. Once the guardrail is recovered, you can continue executing the experiment.
5. Continue executing the shutdown fault injection. If the fault injection is successful, the recovery action will not be automatically executed, and the experiment will not be paused.
Yes
No
Was this page helpful?