- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
- Product Overview
- Purchase Guide
- Quick Start
- Operation Guide
- Fault Action Library
- Compute
- JVM Process CPU at Full Load
- Cross-AZ Experiment in CVM
- CVM DNS Unavailability Experiment
- CVM Domain Name Parsing Tampering Experiment
- CVM System Time Skew
- CVM Disk IO Hang Fault Experiment
- CVM Memory OOM and Disk IO Load
- Experiments on CVM Intra-host Network Disorder
- CVM Kernel Faults
- Experiments on CVM Intra-host Network Latency
- High Utilization of CVM Resources (CPU, Memory, and Disk)
- Experiments on CVM Intra-host Network Corruption
- Experiments on CVM Intra-host Network Duplication
- Experiments on CVM Intra-host Network Occupation
- CVM Network Interruption
- CVM Ping Unreachable
- Cloud API Ban in CVM
- Database
- Primary-secondary Switch in TencentDB for PostgreSQL
- MySQL Instance Overall Unavailable
- Primary-secondary Switch in MySQL
- Setting Maximum Number of Connections in MySQL
- Primary Node Fault Experiment on TencentDB for MySQL
- TencentDB for MySQL Read-only Instance Group Unavailable
- Primary-secondary Switch in TDSQL for MySQM
- Primary-secondary Switch in MariaDB
- Primary-secondary Switch in TDSQL-C
- Practice of TencentDB for Redis Proxy Node Faults
- TencentDB for Redis Primary Node Fault
- Primary-secondary Nodes in TencentDB for Redis Instance Unavailable
- Simulating Primary-secondary Switch in Redis
- Simulating MongoDB Storage Node Fault
- Simulating Self-Built MySQL Crash Through Network Blocking
- Restarting TencentDB for MySQL
- Primary-secondary Switch in SQL Server
- Network
- Container
- Simulating Container Resource Network Faults
- Experiment on Container Resource Pod Operation Faults
- Experiment on Container Resource Node Faults
- Experiment on Container Resource Application Process Faults
- Standard Cluster and Serverless Cluster Super Node Faults
- Serverless Pod Fault Experiment Case
- Cluster Node Resource (CPU, Memory, Disk) Stress Test Faults
- Serverless Pod Virtual Node Shutdown Faults
- Simulating Serverless Cluster Pod Network Faults
- High Cluster Pod Resource (CPU, Memory, and Disk) Utilization Rate
- Cluster Pod TencentCloud API Ban
- Big Data
- Cloud Load Balancer
- Message Queue
- Direct Connect
- Custom Actions
- Cloud Streaming Services (CSS)
- Compute
- Permission Management Guide
- API Documentation
- FAQs
- Related Protocol
- Contact Us
Background
In intra-city double site active-active or cross-region multi-site active-active disaster recovery scenarios, disaster recovery resources are typically deployed in different availability zones or regions from the main resources, each belonging to different subnets. When a fault occurs at the availability zone or regional level, a disaster recovery switch can be initiated. To verify the effectiveness of your disaster recovery architecture, you can use the CFG VPC Subnet Network Isolation action to block the subnet where the main resources are located, simulating the inaccessibility of resources due to a fault.
Fault Effect
The VPC subnet isolation fault provides two ways of network isolation: Single Subnet and All Subnets.
Single Subnet
When the fault action parameter Isolation Scope is set to Single Subnet, the inbound and outbound traffic of the selected subnet will be blocked, and access between subnets will also be restricted. For example, as shown in the figure below, when the Single Subnet network isolation fault is injected into Subnet A and Subnet B, all the inbound and outbound traffic of Subnet A and Subnet B will be blocked. Subnet A cannot access Subnet B or other subnets, but internal traffic within each subnet will not be affected.
All Subnets
When the fault action parameter Isolation Scope selects All Subnets, the inbound and outbound traffic of the selected subnets will be blocked, but access between the selected subnets will not be affected. For instance, as shown in the figure below, when the All Subnets network isolation fault is injected into Subnet A and Subnet B, internal traffic within Subnet A and Subnet B remains unaffected, but traffic attempting to access other subnets will be blocked. You can inject faults into subnets within the same availability zone to simulate a network isolation fault between that availability zone and other availability zones.
Experiment Preparation
Create two subnets under the same VPC, and associate private network CLB, CVM, and MySQL resources with these subnets. The network topology is as follows: the primary availability zone is associated with two CVMs, one private network gateway, and one MySQL instance, while the replica availability zone is associated with one CVM and one private network gateway. Since instances under the same VPC are interconnected via the network by default, resources between subnets can access each other before fault injection. When the primary availability zone subnet is blocked, instances within the primary availability zone subnet can still access each other normally, but external access to instances in the primary availability zone subnet will fail.
Note:
Subnet network isolation is achieved by setting network ACL rules on the subnet, and any existing persistent connections will be immediately disconnected.
If the target subnet has existing network ACL rules, those rules will be temporarily unbound during fault injection and re-bound during recovery. Do not manually modify or delete network ACL rules during the experiment.
Subnet network isolation cannot be used to simulate a database single availability zone fault. For database single availability zone fault, see Database-Related Failures action.
When the Isolation Scope is set to All Subnets, a maximum of 20 subnets can be added. When you need to isolate all subnets for more than 20 subnets, please contact Tencent Cloud Assistant. Submit a Ticket.
This fault action does not block external Ping detection to CLBs within the subnet.
Experiment Implementation
Step 1: Create an Experiment
1. Log in to the Tencent Cloud Smart Advisor > Chaotic Fault Generator Console.
2. In the left sidebar, select the Experiment Management page, click Create a New Experiment, and select to create a blank experiment.
3. After filling in the basic information, enter the Experiment Object Configuration. Select the VPC Subnet under Network for the object type, and click Add Instance.
Click Add Instance, and a list of all VPC subnet information in the target region will be displayed. Filter the subnets based on subnet ID, VPC instance ID, tags, and availability zone keys.
Note:
The subnet network isolation has a wide impact range. Please carefully select the scope of the fault injection instances.
4. After you select the target subnets, click Add Now to add the experiment actions.
Select Network isolation for the experiment action, and then click Next.
Set the Isolation Scope parameter and click Confirm.
5. After you click Confirm, you can see that the fault action will automatically bring up the recovery action.
6. Click Next to enter the Global Configuration. For global configuration details, see Quick Start. Note that subnets do not have corresponding basic resource monitoring metrics. After you confirm, click Submit, and the platform will automatically perform an environment pre-check.
Step 2: Execute the Experiment
In the Experiment Action Group, click Execute to execute an experiment. Since the experiment is manual, manually execution of the fault action is needed.
Step 3: Verify the Injection Effect
In the Virtual Private Cloud console, under the Network Topology Map menu, you can see that the Network ACL rules have been added to the corresponding subnets.
For testing instance access, it is expected that access between instances in the subnets within the faulty availability zone will not be affected, but access from outside the subnet to instances in the faulty subnet fails.
CVM accessing CVM
Normal access within the same subnet
Failure when accessing different subnets (command blocked)
CVM accessing MySQL
Normal access within the same subnet
Failure access within different subnets
CVM accessing CLB
Since CLB uses VPCGW for Ping detection, the subnet network isolation will not block Ping detection. You can use telnet to detect the service-side ports.
Normal access within the same subnet
Failure when accessing different subnets (command blocked)
Step 4: Execute Fault Recovery Actions
Click Execute Recovery Action and wait for the action to be executed successfully.
Step 5: Verify the Recovery Effect
See Step 3 for verification. The expected result is that instances within the subnet and between subnets should be accessed normally.
Yes
No
Was this page helpful?