Primary Node Fault Experiment on TencentDB for MySQL
Last updated: 2024-09-26 15:47:38
Background
CFG provides a fault action for simulating primary node faults in TencentDB for MySQL. Disaster recovery and overall high availability of your business MySQL can be verified through the fault action.
Note:
This fault action requires that your TencentDB for MySQL instance be a multi-node architecture. If this condition is not satisfied, the fault action will not be executed. You can go to TencentDB for MySQL Console to check instance architecture type and adjust instance architecture.
Fault Description
This fault will inject a fatal error into the instance to simulate a primary node fault in a multi-node architecture instance. During fault action execution, temporary database disconnection or failure in connection may occur, which will result in the database access failure. Prudence is required during operation. After fault injection, MySQL instance will perform primary-secondary switch, the original secondary node will become a new primary node, and a new node will be selected from the original primary availability zone as a new secondary node.
Experiment Implementation
Step 1: Experiment Preparation
A TencentDB for MySQL instance with a dual-node architecture.
A CVM instance used for testing MySQL instance connection.
3. Fill in experiment information and select TencentDB for MySQL instance for Experiment Objective.
4. Add MySQL primary node fault experiment action. Click Next.
5. Configure action parameter information for fault actions and click Confirm.
6. After the configuration of action parameters, click Next. After confirming all configurations, click Submit to complete experiment creation.
Step 3: Experiment Execution
During fault execution, a primary node fault in MySQL instance will be triggered, and a primary-secondary switch will occur. An alarm can be received and changes in primary-secondary node architectures can be observed from TencentDB for MySQL Console.
Before a Fault Occrs:
Observe the availability zone where primary-secondary nodes of MySQL instance are located. Log in to current MySQL instance through CVM instance and create a client connection.
Click One-key Diagnosis in the upper right corner, go to DBbrain, connections currently created by CVM can be checked.
Execute an Experiment:
In Experiment Details page, click Execute in Experiment Action Group to start executing an experiment. After execution of a fault action, go to TencentDB for MySQL console to check alarms and node changes.
After a successful fault execution, you can check fault execution results through TencentDB for MySQL console. That is to say, the original replica node is promoted to primary node after the fault, and a new node is selected as replica node from the availability zone where the original primary node is located; and you can receive an abnormality alarm in the upper right corner (data is from TencentDB for DBbrain inspection).
After a fault occurs in MySQL primary node, you can see through the instance monitoring panel that the current instance is temporarily unavailable when there is a fault in the primary node.
In such period, execute SQL through CVM, and the original connection has been interrupted and reconnection is required.
Meanwhile, the original connection monitored through DBbrain session no longer exists and a new connection appears.
Yes
No
Was this page helpful?