- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Development Guide
- Design of Data Table
- Importing Data
- Import Overview
- Import Methods
- Synchronizing Data with JDBC
- Synchronizing Data Through External Tables
- Real-time or Batch Writing of MySQL Data
- Importing Data from Kafka
- Importing with DataX
- Importing Doris from Logstash
- Data Transformation, Column Mapping and Filtering
- Importing Strict Mode
- Importing Data in JSON Format
- Exporting Data
- Basic Feature
- Query Optimization
- Ecological Expansion Feature
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Cluster Operation APIs
- ModifyClusterConfigs
- ModifyInstanceKeyValConfigs
- ModifyNodeStatus
- RestartClusterForConfigs
- RestartClusterForNode
- CreateInstanceNew
- DestroyInstance
- ModifySecurityGroups
- ReduceInstance
- ResizeDisk
- ScaleOutInstance
- ScaleUpInstance
- DescribeClusterConfigsHistory
- ModifyInstance
- DescribeCreateTablesDDL
- DescribeInstanceOperationHistory
- Database and Table APIs
- Cluster Information Viewing APIs
- Hot-Cold Data Layering APIs
- Database and Operation Audit APIs
- CancelBackupJob
- CreateBackUpSchedule
- DeleteBackUpData
- DescribeBackUpJob
- DescribeBackUpJobDetail
- DescribeBackUpTables
- DescribeBackUpTaskDetail
- DescribeRestoreTaskDetail
- DescribeBackUpSchedules
- RecoverBackUpJob
- DescribeInstanceOperations
- DescribeDatabaseAuditDownload
- DescribeDatabaseAuditRecords
- DescribeSlowQueryRecords
- DescribeSlowQueryRecordsDownload
- DescribeQueryAnalyse
- User and Permission APIs
- Resource Group Management APIs
- Data Types
- Error Codes
- Cloud Ecosystem
- Practical Tutorial
- Performance Testing
- FAQs
- Service Level Agreement
- Contact Us
- Glossary
- Product Policy
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Development Guide
- Design of Data Table
- Importing Data
- Import Overview
- Import Methods
- Synchronizing Data with JDBC
- Synchronizing Data Through External Tables
- Real-time or Batch Writing of MySQL Data
- Importing Data from Kafka
- Importing with DataX
- Importing Doris from Logstash
- Data Transformation, Column Mapping and Filtering
- Importing Strict Mode
- Importing Data in JSON Format
- Exporting Data
- Basic Feature
- Query Optimization
- Ecological Expansion Feature
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Cluster Operation APIs
- ModifyClusterConfigs
- ModifyInstanceKeyValConfigs
- ModifyNodeStatus
- RestartClusterForConfigs
- RestartClusterForNode
- CreateInstanceNew
- DestroyInstance
- ModifySecurityGroups
- ReduceInstance
- ResizeDisk
- ScaleOutInstance
- ScaleUpInstance
- DescribeClusterConfigsHistory
- ModifyInstance
- DescribeCreateTablesDDL
- DescribeInstanceOperationHistory
- Database and Table APIs
- Cluster Information Viewing APIs
- Hot-Cold Data Layering APIs
- Database and Operation Audit APIs
- CancelBackupJob
- CreateBackUpSchedule
- DeleteBackUpData
- DescribeBackUpJob
- DescribeBackUpJobDetail
- DescribeBackUpTables
- DescribeBackUpTaskDetail
- DescribeRestoreTaskDetail
- DescribeBackUpSchedules
- RecoverBackUpJob
- DescribeInstanceOperations
- DescribeDatabaseAuditDownload
- DescribeDatabaseAuditRecords
- DescribeSlowQueryRecords
- DescribeSlowQueryRecordsDownload
- DescribeQueryAnalyse
- User and Permission APIs
- Resource Group Management APIs
- Data Types
- Error Codes
- Cloud Ecosystem
- Practical Tutorial
- Performance Testing
- FAQs
- Service Level Agreement
- Contact Us
- Glossary
- Product Policy
This document introduces how to use the Star Schema data set to perform performance testing on Tencent Cloud TCHouse-D, and provides a reference scheme for data import and performance testing.
About SSB Performance Test
SSB (Star Schema Benchmark) is a lightweight performance test set for data warehouse scenes. SSB provides a simplified star model data set based on TPC-H, which is mainly used to test the performance of multi-table join queries under the star model. In addition, the industry practice is to flatten SSB into a wide table model (hereinafter referred to as: SSB FLAT) to test the performance of the query engine.
Test Scheme Introduction
Test Environment Preparation
Hardware Environment
In the reference scheme given in this document, the tested cluster includes 1 FE and 3 BEs. The FE/BE node processes are deployed separately. The specific specifications are as follows. It should be noted that in actual testing, such a large amount of hardware resources will not be consumed.
Node Type | Node Specifications |
1 FE, standard | CPU:4 cores Memory: 16 GB Hard disk: Enhanced SSD Cloud Disk 200 GB |
3 BEs, standard | CPU:16 cores Memory: 64 GB Hard disk: Enhanced SSD Cloud Disk 1000 GB |
Software Version
Tencent Cloud TChouse-D 1.2.7
Test Script Preparation
Test SSB data set.
Generate data set.
sh bin/gen-ssb-data.sh -s 100 -c 100277M /data/ssb-tools/bin/ssb-data//customer.tbl228K /data/ssb-tools/bin/ssb-data//date.tbl5.9G /data/ssb-tools/bin/ssb-data//lineorder.tbl.17.7G /data/ssb-tools/bin/ssb-data//lineorder.tbl.107.6G /data/ssb-tools/bin/ssb-data//lineorder.tbl.27.2G /data/ssb-tools/bin/ssb-data//lineorder.tbl.36.0G /data/ssb-tools/bin/ssb-data//lineorder.tbl.46.0G /data/ssb-tools/bin/ssb-data//lineorder.tbl.56.0G /data/ssb-tools/bin/ssb-data//lineorder.tbl.66.0G /data/ssb-tools/bin/ssb-data//lineorder.tbl.77.4G /data/ssb-tools/bin/ssb-data//lineorder.tbl.87.5G /data/ssb-tools/bin/ssb-data//lineorder.tbl.9116M /data/ssb-tools/bin/ssb-data//part.tbl17M /data/ssb-tools/bin/ssb-data//supplier.tbl
Under the -s 100 parameter, the size of the data set generated is:
SSB Table Name | Number of rows | Remarks |
LINEORDER | 600,037,902 | Product Order Details Table |
CUSTOMER | 3,000,000 | Customer Information Table |
PART | 1,400,000 | Parts Information Table |
SUPPLIER | 200,000 | Supplier Information Table |
DATE | 2,556 | Date Table |
Create a table
Modify the doris-cluster.conf configuration file vim conf/doris-cluster.conf.
Modify configuration: FE_HOST, PASSWORD, DB.
# cat doris-cluster.conf# Any of FE hostexport FE_HOST='127.0.0.1'# http_port in fe.confexport FE_HTTP_PORT=8030# query_port in fe.confexport FE_QUERY_PORT=9030# Doris usernameexport USER='root'# Doris passwordexport PASSWORD=''# The database where SSB tables locatedexport DB='ssb_100g'# The scale of testing dataexport SCALE='100g' # only support '100g' or '1t'
Create a Table:
sh bin/create-ssb-tables.sh
Import Data
sh bin/load-ssb-data.sh
Check Imported Data
selectcount(*)from part;selectcount(*)from customer;selectcount(*)from supplier;selectcount(*)fromdate;selectcount(*)from lineorder;selectcount(*)from lineorder_flat;
The amount of data shall be consistent with the number of rows of generated data.
Query
Query SSB tables.
# sh bin/run-ssb-queries.shq1.1: 46q1.2: 29q1.3: 26q2.1: 340q2.2: 273q2.3: 257q3.1: 542q3.2: 237q3.3: 297q3.4: 57q4.1: 732q4.2: 372q4.3: 483total time: 3691 ms
Query FLAT tables.
# sh bin/run-ssb-flat-queries.shq1.1: 26q1.2: 10q1.3: 35q2.1: 85q2.2: 83q2.3: 60q3.1: 157q3.2: 78q3.3: 75q3.4: 13q4.1: 131q4.2: 49q4.3: 33total time: 835 ms
Thus, SSB data generation, table creation, import, and query are completed.
Yes
No
Was this page helpful?