tencent cloud

Feedback

DocumentationTDMQ for CKafkaGeneral ReferencesConducting Production and Consumption Pressure Testing on CKafka
DocumentationTDMQ for CKafkaGeneral ReferencesConducting Production and Consumption Pressure Testing on CKafka

Conducting Production and Consumption Pressure Testing on CKafka

Last updated: 2024-01-09 15:02:47

Testing Tool

The open-source script of the Kafka client can be used for Kafka producer and consumer performance testing. Test results are displayed mainly based on the size of messages sent per second (MB/second) and the number of messages sent per second (records/second).
Kafka producer test script: $KAFKA_HOME/bin/kafka-producer-perf-test.sh
Kafka consumer test script: $KAFKA_HOME/bin/kafka-consumer-perf-test.sh

Testing Command

Note:
The ckafka vip:vport in the following sample commands should be replaced by the actual IP and port assigned for your instance.
Sample command for production testing:
bin/kafka-producer-perf-test.sh
--topic test
--num-records 123
--record-size 1000
--producer-props bootstrap.servers= ckafka vip : port
--throughput 20000
Sample command for consumption testing:
bin/kafka-consumer-perf-test.sh
--topic test
--new-consumer
--fetch-size 10000
--messages 1000
--broker-list bootstrap.servers=ckafka vip : port

Suggestions

We recommend you create three or more partitions to increase throughput. This is because there must be at least three CKafka cluster nodes at the backend. If only one partition is created, it will be distributed in a single broker, which will affect CKafka performance.
As messages in each CKafka partition are ordered, the production performance will be affected if there are too many partitions. We recommend you create up to six partitions.
It is necessary to simulate concurrency with multiple clients to ensure the testing effect. We recommend you use multiple test servers as the pressure test clients (producers) and start multiple pressure test programs on each test server to increase concurrency. To avoid the high load of test servers, you are also recommended to start one producer every second rather than all producers simultaneously.

Catalog

In The Article

Conducting Production and Consumption Pressure Testing on CKafka

Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support