The performance test in this document is conducted with the TPC-C benchmark load built in HammerDB. TPC-C is a typical OLTP workload that simulates a scenario where a wholesaler with multiple warehouses ships goods to a large number of customers. The adjustment of the number of warehouses can reflect the data size that the database can sustain in the test.
The test instances are of 2008 R2 Enterprise Edition, 2012 Enterprise Edition, 2014 Enterprise Edition, 2016 Enterprise Edition, 2017 Enterprise Edition, and 2019 Enterprise Edition, and 2022 Enterprise Edition.
The test single-node (formerly Basic Edition) instances cover all purchasable specifications, including 1-core 2 GB, 1-core 4 GB, 2-core 4 GB, 2-core 8 GB, 4-core 8 GB, 4-core 16 GB, 8-core 16 GB, 8-core 32 GB, 16-core 32 GB, 16-core 64 GB, 24-core 48 GB, and 24-core 96 GB.
Load generation environment
The machines on which HammerDB is installed are of the same models as the database instances, ensuring that the performance of the SQL Server instances can be fully measured in the stress test.
TPC-C benchmark parameters
Number of Warehouses = 100: Sets the number of warehouses to 100.
Minutes of Rampup Time = 2: Sets the warm-up time before the test to 2 minutes.
Minutes Test Duration = 5: Sets the test duration to 5 minutes.
Number of virtual users
The number of virtual users is the number of concurrent connections. In this document, different numbers of concurrent connections are tested on instances of different editions with different specifications.
Two-node (formerly High Availability Edition)
Concurrent Connections
2
4
8
16
32
64
128
256
512
1024
1-core 2 GB
✓
✓
✓
✓
✓
✓
✓
✓
-
-
1-core 4 GB
✓
✓
✓
✓
✓
✓
✓
✓
-
-
1-core 8 GB
✓
✓
✓
✓
✓
✓
✓
✓
-
-
2-core 4 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
2-core 8 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
2-core 16 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
4-core 8 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
4-core 16 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
4-core 32 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
8-core 16 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
8-core 32 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
8-core 64 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
12-core 24 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
12-core 48 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
12-core 96 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
16-core 32 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
16-core 64 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
16-core 128 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
24-core 48 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
24-core 96 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
24-core 192 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
32-core 64 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
32-core 128 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
32-core 256 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
48-cores 96 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
48-core 192 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
48-core 384 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
64-core 128 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
64-core 256 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
64-core 512 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
90-core 720 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Single-node (formerly Basic Edition)
Concurrent Connections
2
4
8
16
32
64
128
256
512
1,024
1-core 2 GB
✓
✓
✓
✓
✓
✓
✓
✓
-
-
1-core 4 GB
✓
✓
✓
✓
✓
✓
✓
✓
-
-
2-core 4 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
2-core 8 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
4-core 8 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
4-core 16 GB
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
8-core 16 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
8-core 32 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
-
16-core 32 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
16-core 64 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
24-core 48 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
24-core 96 GB MEM
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Test Method
1. Prepare the TPC-C workload.
Number of Warehouses: The number of warehouses, which will affect the size of the test database generated.
Virtual Users to Build Schema: The number of concurrent connections when generating the load data, which cannot be larger than the number of warehouses. This value affects the efficiency of load data generation, so it is recommended to be the same as the number of CPU cores of the load generating device.
2. Set the test script.
Total Transactions per User: The total number of transactions per user. We recommend that you set this parameter to a higher value so as to ensure that the user will not exit due to the completion of transactions during the stress test.
Minutes of Rampup Time: Warm-up time for the stress test.
Minutes for Test Duration: Duration of the stress test.
3. Set the automated test script.
Minutes per Test in Virtual User Sequence: The interval between two automated test sessions during which the program completes various tasks such as creating virtual users, warming up, running the test, and stopping the test. This value should be greater than the sum of "Minutes of Rampup Time" and "Minutes for Test Duration".
Active Virtual User Sequence (Space Separated): The number of virtual users generated by each iteration of the automated test. It can be understood as the number of concurrent connections.
4. Select Autopilot > Autopilot in the left pane to start the test.
5. The test result will be output in the hammerdb.log file.
Test Results
Note:
The TPM in HammerDB is obtained through the SQL Server performance counter "batch requests/sec", so the TPM actually refers to the batch requests per minute.
The size of test data set for a instance specification is larger than the memory size of the specification.
Due to potential fluctuations in each stress test, the following results should be considered as references only.
Two-node (Enhanced SSD Cloud Block Storage)
Two-node (Enhanced SSD Cloud Block Storage) performance comparison trend chart for each version
Two-node (Enhanced SSD Cloud Block Storage) TPM performance comparison value for each version
