Message Classification
In the message queue, based on the characteristics and use cases of messages, messages can be classified as follows:
|
Normal Messages | No Order | Best | Huge throughput with no requirements for production and consumption orders |
Local Ordered Messages | All messages in the same partition follow the First In, First Out (FIFO) rule. | Better | High throughput, ordered within the same partition, and unordered across different partitions |
Global Ordered Messages | All messages in the same Topic follow the First In, First Out (FIFO) rule. | General | General throughput, globally ordered, and single partition |
Dead Letter Messages | - | - | Messages that cannot be consumed normally. |
Scheduled Messages | Messages are consumed after being sent to the server and delayed until a certain time point. | There is an error of about 1 second. | Consume the message at a specified time point. |
Delayed Messages | Messages are consumed after being sent to the server and delayed for a period of time. | There is an error of about 1 second. | Consume the message after a specified time interval. |
Normal Messages
Normal message is a basic message type, where a message is delivered to the specified Topic by the producer and then consumed by the consumer subscribed to the Topic. There is no concept of order in the Topic of a normal message, and multiple partitions can be used to improve the efficiency of message production and consumption, and its performance is best when the throughput is huge.
Local Ordered Messages
Compared with normal messages, local ordered messages have an additional characteristic of local order. Within the same partition, consumers consume messages strictly in the order they are delivered to the partition by the producer. While ensuring a certain order, local ordered messages retain the partition mechanism to improve performance. However, they cannot guarantee the order across different partitions.
Global Ordered Messages
The most distinctive characteristic of globally sequential messages is that they ensure messages are consumed in the strict order they are delivered by the producer. Therefore, it uses a single partition to process messages, and users cannot customize the number of partition. Compared with the other two message types, this type has lower performance.
Dead Letter Messages
A dead letter message is a message that cannot be consumed normally. The TDMQ for Apache Pulsar will automatically create a dead letter queue to process such messages when creating a new subscription (when a consumer subscribes to a Topic).
Scheduled Messages
After a message is sent to the server, the actual business does not want the consumer to receive the message immediately, but rather delay it to a certain point in time. These messages are collectively referred to as scheduled messages, which can be seen as a special usage of delayed messages, achieving the same final effect as delayed messages.
Delayed Messages
After a message is sent to the server, the actual business does not want the consumer to receive the message immediately but rather delay it for a period of time. These messages are collectively referred to as delayed messages.
Retry Queue
A retry queue is designed to ensure that messages are consumed normally. If no normal response is received after a message is consumed by the consumer for the first time, it will enter the retry queue, and when the retries reach a certain number of times, the retries are stopped and delivered to the dead letter queue.
In actual cases, messages may not be processed promptly due to temporary issues such as network jitter and service restart, and the retry mechanism of the retry letter topic can be a good solution in this case.
Dead Letter Queue
A dead letter queue is a special type of message queue used to centrally process messages that cannot be consumed normally. If a message cannot be consumed after a specified number of retries in the retry queue, the TDMQ for Apache Pulsar will determine that the message cannot be consumed under the current situation and deliver it to the dead letter queue.
In actual cases, messages may not be consumed due to prolonged service downtime or network disconnection. In such cases, messages will not be discarded immediately; instead, they will be persisted for a longer period in the dead letter queue. Once a solution is found, users can create a consumer subscription to the dead letter queue.
Was this page helpful?