- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
Practice of TencentDB for Redis Proxy Node Faults
Last updated: 2024-09-26 15:47:38
Background
TencentDB for Redis allows automatic read-write separation through the Proxy layer. Proxy node can determine whether replica read-only mode is to be enabled. Proxy layer acts as a proxy for Redis sharded cluster. A fault in Proxy layer will have direct impact on access to data in TencentDB for Redis. Users can use the TencentDB for Redis Proxy node fault action provided by the platform to check if a fault in Proxy layer node will have impact on their businesses.
Experiment Implementation
Step 1: Experiment Instance Preparation
Select one or more TencentDB for Redis instances as instance objects of an experiment.
Step 2: Experiment Orchestration
1. Log in to the Tencent Smart Advisor > Chaotic Fault Generator, go to the Experiment Management page, and click Create a New Experiment.
2. Click Skip and create a blank experiment. Fill in experiment information, select Redis Memory Edition under Cloud Resources for Resource Object, add target instance.
3. Click Add Now to add an experiment action, select Redis Proxy node failure , and click Next.
4. Fill in corresponding action parameters, and click Confirm.
Note:
There are two action parameters required for this action: fault injection availability zone and fault node proportion (%).
Fault Injection Availability Zone:
Primary Availability Zone: This means that a Proxy node where a fault is to be injected will be selected from primary availability zone.
Secondary Availability Zone: This means that a Proxy node where a fault is to be injected will be selected from secondary availability zone.
All Availability Zones: This means that a Proxy node where a fault is to be injected will be selected from all availability zones.
Faulty Node Ratio (%) : This parameter is an integer ranging 1~100, indicating that a certain number of proxy nodes will be selected for fault injection.
For example: Parameters for a certain experiment can be so configured that fault is injected into 50% of Proxy nodes in the primary availability zone. If the total number of proxy nodes in the primary availability zone in the instance is five, 5 * 50% = 3 (rounded up) nodes will be randomly selected from the 5 proxy nodes in the primary availability zone for fault injection.
5. After the configuration of action parameters, click Next. After confirming all configurations, click Submit to complete experiment creation.
Step 3: Experiment Execution
1. Initial state of a TencentDB for Redis Proxy node. You can go to TencentDB for Redis console, click corresponding instances and select System Monitoring > Monitoring Metrics > Instance Monitoring > Proxy Monitoring.
2. To execute an experiment, click Action Card and observe the results of execution.
It can seen that a fault has been injected into the specified node and the node has crashed. And then there will be a new Proxy node.
[2023-02-02 10:23:10]: General Parameters Configuration:[2023-02-02 10:23:10]: Lead waiting time (s): 0[2023-02-02 10:23:10]: Action timeout (s): 1800[2023-02-02 10:23:10]: Post waiting time (s): 0[2023-02-02 10:23:10]: Custom parameters configuration:[2023-02-02 10:23:10]: Fault node proportion (%): 60[2023-02-02 10:23:10]: Fault injection availability zone: primary availability zone[2023-02-02 10:23:10]: Start execution [Redis Proxy Node Fault] action, instances involved: ['crs-xxxxxxx'][2023-02-02 10:23:11]: [Redis Proxy node fault] action has been executed, and monitoring on returned results of action is started.[2023-02-02 10:23:14]: [Redis Proxy Node Fault] action call [Successful], details:Instance: [crs-xxxxxxx] Execution result: [Successful] Details: Instance: crs-xxxxxxxx, fault injection into CRS Proxy node is successful, injected Proxy node object ID = ['d06d2ffb1b600ffa877137c1c5c5265221cd55fa', '1ce5fec58407759a9ff04c30c1a9e485911ac25d']
Yes
No
Was this page helpful?