- Release Notes and Announcements
- Release Notes
- Announcements
- Announcement on Release of New Upgrade Mechanism for Kernel Minor Version
- Announcement on Optimization of Instance Storage Capacity, IOPS, and I/O Bandwidth
- Announcement on Some APIs with CAM Authentication Integrated
- Announcement on Sales Suspension of Some General Specifications
- Announcement on Some APIs Integrated with CAM Authentication
- Announcement on Some APIs with CAM Authentication Integrated
- Announcement on Change of Part of Parameters
- Announcement on Update of Database Audit Feature
- Announcement on Authentication of a Newly Added API Interface of Database Audit
- Announcement on Change of Public Network Linkage
- TDSQL-C for MySQL Audit Service Update
- Announcement on Supplementary Charge for Clusters with Pay-as-You-Go Storage Billing Mode
- Announcement on New APIs Integrated with CAM Permissions
- Announcement on the Release of New Version of Database Proxy
- Emergency Fix for Certain Monitoring Metrics
- Monitoring Metric Optimization
- Change to Database Audit Resource ID
- Ops Change
- Storage Price Adjustment Notification
- Beginner's Guide
- Product Introduction
- Kernel Features
- Kernel Overview
- Kernel Version Release Notes
- Optimized Kernel Version
- Functionality Features
- Performance Features
- Columns Syntax
- Correlated Subquery Result Cache
- Global Index
- Binlog Write Optimization
- Large Transaction Binlog Optimization
- Binlog Purging Performance Optimization
- Optimization of Plan Caching Point Query
- Auto-Increment Column Persistence
- Invisible Index
- Computation Pushdown
- Parallel Initialization of InnoDB Buffer Pool
- Security Features
- Stability Feature
- Purchase Guide
- Getting Started
- Database Audit
- Serverless Service
- Operation Guide
- Switching Cluster Page View in Console
- Database Connection
- Instance Management
- Configuration Adjustment
- Cluster Management
- Scaling Instance
- Database Proxy
- Database Proxy Overview
- Use Limits
- Database Proxy Kernel Features
- Managing Database Proxy
- Enabling Database Proxy
- Setting Database Proxy Address
- Modifying or Deleting Connection Addresses
- Viewing and Changing the Access Policy
- Rebalancing the Load
- Transaction Split Feature
- Access Mode
- Adjusting Database Proxy Configuration
- Switching Database Proxy Network
- Viewing Database Proxy Monitoring Data
- Disabling Database Proxy
- Automatic Read/Write Separation
- Connection Pool Feature
- Other Features
- Account Management
- Database Management
- Database Management Tool
- Columnar Storage Index (CSI)
- Parameter Configuration
- Multi-AZ Deployment
- Backup and Restoration
- Operation Log
- Data Migration
- Parallel Query
- Database Security and Encryption
- Monitoring and Alarms
- Basic SQL Operations
- Connecting to TDSQL-C for MySQL Through SCF
- Tag
- Practical Tutorial
- Classified Protection Practice for Database Audit of TDSQL-C for MySQL
- Upgrading Database Version from MySQL 5.7 to 8.0 Through DTS
- Usage Instructions for TDSQL-C MySQL
- New Version of Console
- Implementing Multiple RO Groups with Multiple Database Proxy Connection Addresses
- Strengths of Database Proxy
- Selecting Billing Mode for Storage Space
- Creating Remote Disaster Recovery by DTS
- Creating VPC for Cluster
- Data Rollback
- Solution to High CPU Utilization
- How to Authorize Sub-Users to View Monitoring Data
- White Paper
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Billing APIs
- Instance APIs
- CloseClusterPasswordComplexity
- CopyClusterPasswordComplexity
- DescribeClusterDetail
- DescribeClusterPasswordComplexity
- ModifyClusterPasswordComplexity
- OpenClusterPasswordComplexity
- UpgradeClusterVersion
- UpgradeInstance
- SetRenewFlag
- OfflineCluster
- ModifyMaintainPeriodConfig
- IsolateInstance
- IsolateCluster
- DescribeMaintainPeriod
- DescribeInstanceSpecs
- DescribeClusters
- CreateClusters
- AddInstances
- OfflineInstance
- DescribeInstanceDetail
- DescribeClusterInstanceGrps
- DescribeInstances
- ActivateInstance
- ModifyInstanceName
- ModifyClusterName
- SearchClusterTables
- SearchClusterDatabases
- RestartInstance
- Multi-AZ APIs
- Account APIs
- Audit APIs
- Database Proxy APIs
- Backup and Restoration APIs
- Parameter Management APIs
- Performance Analysis APIs
- Serverless APIs
- ResourcePackage APIs
- Other APIs
- CloseWan
- CreateClusterDatabase
- DeleteClusterDatabase
- DescribeClusterDetailDatabases
- ModifyClusterDatabase
- OpenWan
- SwitchClusterVpc
- OpenReadOnlyInstanceExclusiveAccess
- OpenClusterReadOnlyInstanceGroupAccess
- ModifyDBInstanceSecurityGroups
- DescribeProjectSecurityGroups
- DescribeDBSecurityGroups
- SwitchProxyVpc
- DescribeFlow
- DescribeZones
- ModifyVipVport
- Data Types
- Error Codes
- FAQs
- Service Agreement
- TDSQL-C Policy
- General References
- Glossary
- Contact Us
- Release Notes and Announcements
- Release Notes
- Announcements
- Announcement on Release of New Upgrade Mechanism for Kernel Minor Version
- Announcement on Optimization of Instance Storage Capacity, IOPS, and I/O Bandwidth
- Announcement on Some APIs with CAM Authentication Integrated
- Announcement on Sales Suspension of Some General Specifications
- Announcement on Some APIs Integrated with CAM Authentication
- Announcement on Some APIs with CAM Authentication Integrated
- Announcement on Change of Part of Parameters
- Announcement on Update of Database Audit Feature
- Announcement on Authentication of a Newly Added API Interface of Database Audit
- Announcement on Change of Public Network Linkage
- TDSQL-C for MySQL Audit Service Update
- Announcement on Supplementary Charge for Clusters with Pay-as-You-Go Storage Billing Mode
- Announcement on New APIs Integrated with CAM Permissions
- Announcement on the Release of New Version of Database Proxy
- Emergency Fix for Certain Monitoring Metrics
- Monitoring Metric Optimization
- Change to Database Audit Resource ID
- Ops Change
- Storage Price Adjustment Notification
- Beginner's Guide
- Product Introduction
- Kernel Features
- Kernel Overview
- Kernel Version Release Notes
- Optimized Kernel Version
- Functionality Features
- Performance Features
- Columns Syntax
- Correlated Subquery Result Cache
- Global Index
- Binlog Write Optimization
- Large Transaction Binlog Optimization
- Binlog Purging Performance Optimization
- Optimization of Plan Caching Point Query
- Auto-Increment Column Persistence
- Invisible Index
- Computation Pushdown
- Parallel Initialization of InnoDB Buffer Pool
- Security Features
- Stability Feature
- Purchase Guide
- Getting Started
- Database Audit
- Serverless Service
- Operation Guide
- Switching Cluster Page View in Console
- Database Connection
- Instance Management
- Configuration Adjustment
- Cluster Management
- Scaling Instance
- Database Proxy
- Database Proxy Overview
- Use Limits
- Database Proxy Kernel Features
- Managing Database Proxy
- Enabling Database Proxy
- Setting Database Proxy Address
- Modifying or Deleting Connection Addresses
- Viewing and Changing the Access Policy
- Rebalancing the Load
- Transaction Split Feature
- Access Mode
- Adjusting Database Proxy Configuration
- Switching Database Proxy Network
- Viewing Database Proxy Monitoring Data
- Disabling Database Proxy
- Automatic Read/Write Separation
- Connection Pool Feature
- Other Features
- Account Management
- Database Management
- Database Management Tool
- Columnar Storage Index (CSI)
- Parameter Configuration
- Multi-AZ Deployment
- Backup and Restoration
- Operation Log
- Data Migration
- Parallel Query
- Database Security and Encryption
- Monitoring and Alarms
- Basic SQL Operations
- Connecting to TDSQL-C for MySQL Through SCF
- Tag
- Practical Tutorial
- Classified Protection Practice for Database Audit of TDSQL-C for MySQL
- Upgrading Database Version from MySQL 5.7 to 8.0 Through DTS
- Usage Instructions for TDSQL-C MySQL
- New Version of Console
- Implementing Multiple RO Groups with Multiple Database Proxy Connection Addresses
- Strengths of Database Proxy
- Selecting Billing Mode for Storage Space
- Creating Remote Disaster Recovery by DTS
- Creating VPC for Cluster
- Data Rollback
- Solution to High CPU Utilization
- How to Authorize Sub-Users to View Monitoring Data
- White Paper
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Billing APIs
- Instance APIs
- CloseClusterPasswordComplexity
- CopyClusterPasswordComplexity
- DescribeClusterDetail
- DescribeClusterPasswordComplexity
- ModifyClusterPasswordComplexity
- OpenClusterPasswordComplexity
- UpgradeClusterVersion
- UpgradeInstance
- SetRenewFlag
- OfflineCluster
- ModifyMaintainPeriodConfig
- IsolateInstance
- IsolateCluster
- DescribeMaintainPeriod
- DescribeInstanceSpecs
- DescribeClusters
- CreateClusters
- AddInstances
- OfflineInstance
- DescribeInstanceDetail
- DescribeClusterInstanceGrps
- DescribeInstances
- ActivateInstance
- ModifyInstanceName
- ModifyClusterName
- SearchClusterTables
- SearchClusterDatabases
- RestartInstance
- Multi-AZ APIs
- Account APIs
- Audit APIs
- Database Proxy APIs
- Backup and Restoration APIs
- Parameter Management APIs
- Performance Analysis APIs
- Serverless APIs
- ResourcePackage APIs
- Other APIs
- CloseWan
- CreateClusterDatabase
- DeleteClusterDatabase
- DescribeClusterDetailDatabases
- ModifyClusterDatabase
- OpenWan
- SwitchClusterVpc
- OpenReadOnlyInstanceExclusiveAccess
- OpenClusterReadOnlyInstanceGroupAccess
- ModifyDBInstanceSecurityGroups
- DescribeProjectSecurityGroups
- DescribeDBSecurityGroups
- SwitchProxyVpc
- DescribeFlow
- DescribeZones
- ModifyVipVport
- Data Types
- Error Codes
- FAQs
- Service Agreement
- TDSQL-C Policy
- General References
- Glossary
- Contact Us
Overview
In a database, binlog is a log file used to record database operations, including addition, deletion, and modification. The binlog files increase over time, occupying a large disk space, so they should be regularly purged. However, binlog purging may conflict with normal binlog rotation and write transactions. Particularly when there are many binlog files, performance fluctuation is easily caused by binlog purging. Through analysis on the scenarios of purging binlog files, the Tencent Cloud kernel team has optimized the binlog purging performance.
The optimization addresses the LOCK_index conflict between binlog purging and binlog rotation, preventing failure to obtain the lock and continue writing binlogs in binlog rotation.
Currently, when an index lock is held by binlog purging, if binlog rotation is required after the binlog size reaches 256 MB but the lock cannot be obtained, binlog writing will stop until the index lock is obtained. The main optimization for this issue is that if the index lock cannot be obtained, binlog rotation is not performed and binlogs continue to be written into the current binlog file. Attempts will be made constantly to obtain the index lock and binlog rotation will be performed only after the index lock is obtained.
The optimization reduces the volume of information read from binlog files to be purged, thereby reducing locks.
The binlog purging process requires reading the GTID information from all binlogs to be purged and then writing it into the mysql.gtid executed table. Therefore, the exclusive lock of GTID is held during the entire purging period, resulting in that normal transactions cannot obtain the GTID lock. The main optimization for this issue is that GTID information is read only from the last binlog file in the binlog list, reducing the holding time of the GTID lock and decreasing its impact on normal transactions.
The maximum size of a single binlog file has been increased to 10 GB, significantly reducing the number of times of binlog rotation.
The default binlog size is 256 MB. Under a load of 5,000 QPS, binlog rotation is performed approximately every 1.4s, each taking 10-20 ms. Currently, the max_binlog_size can be increased from 256 MB to 10 GB (open-source MySQL supports up to 1 GB). Under the same load, rotation is now performed every 56s, reducing the frequency of obtaining the index lock. The max_binlog_size parameter is set to 1 GB in the current policy and can be further adjusted based on business load requirements.
Supported Versions
Kernel version 3.1.12 or later for TDSQL-C for MySQL 8.0.
Kernel version 2.1.12 or later for TDSQL-C for MySQL 5.7.
Use Cases
This feature is applicable to scenarios where binlog purging may cause lock contention and block binlog writing in case of many binlog files.
Use Instructions
This feature is used with parameters. Set txsql_binlog_rotate_try_lock_index and txsql_binlog_rotate_try_lock_log to ON, and set txsql_binlog_purge_check_file_count to 10 to use this feature. Detailed parameter descriptions are as follows:
Name | Global or Not | Type | Default Value | Value Range | Description |
txsql_binlog_rotate_try_lock_index | Global | bool | ON | ON/OFF | When it is set to ON, if the lock of the index file cannot be obtained in binlog rotation, the current rotation will be abandoned. |
txsql_binlog_rotate_try_lock_log | Global | bool | ON | ON/OFF | When it is set to ON, if the lock of the binlog file cannot be obtained in binlog rotation, the current rotation will be abandoned. |
txsql_binlog_purge_check_file_count | Global | ulonglong | 10 | 0-UINT64_MAX | The binlog purging process requires obtaining the binlog file name from the binlog index file. If there are many binlog files, this operation is time-consuming. Normally, only one binlog file name is required. This parameter restricts the number of file names obtained. 0 indicates no limit. It is generally set to 10. |
Yes
No
Was this page helpful?