Concepts
Scheduled message: After a message is sent to the server, the business may want the consumer to receive it at a later time point rather than immediately. This type of message is called "scheduled message".
Delayed message: After a message is sent to the server, the business may want the consumer to receive it after a period of time rather than immediately. This type of message is called "delayed message".
Actually, scheduled message can be regarded as a special type of delayed message, which is essentially the same thing.
Use Cases
There is no difference between implementing delay from your business code or a third-party component if your system uses a monolithic design. However, if your system has a large distributed architecture with dozens or even hundreds of microservices, implementing the delay logic through the application may result in a variety of issues, and if a node running the delay program fails, the entire delay logic will be affected.
Given this, it will be a good option to deliver delayed messages to a message queue based on their attributes, as the delay period can be calculated in a unified manner, while the retry and dead letter mechanisms ensure that messages will not get lost.
Examples of specific scenarios are as follows:
After a WeChat Red Packet is sent, the producer sends a message with a 24-hour delay. Once 24 hours have passed, the consumer program receives the message and checks whether the user has claimed the Red Packet. If the user has not claimed it, the system refunds the amount to the original account.
After a mini-program places an order for a product, the backend stores a message with a 30-minute delay. Once the time is up, the consumer receives the message and checks the payment status. If payment has not been made, the order is canceled, implementing the logic of canceling unpaid orders after 30 minutes.
When a user sets a message as a to-do on WeChat, a scheduled message can be sent. The server consumes the scheduled message at the designated time to remind the user of the to-do item.
Directions
The TDMQ for Apache Pulsar SDK provides dedicated APIs to implement scheduled and delayed messages.
For a scheduled message, you need to specify a moment to send it.
For a delayed message, you need to specify a period of time as the delay.
Scheduled Message
Scheduled messages are implemented by the producer's deliverAt()
method. Below is the sample code:
String value = "message content";
try {
long timeStamp = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse("2020-11-11 00:00:00").getTime();
MessageId msgId = producer.newMessage()
.value(value.getBytes())
.deliverAt(timeStamp)
.send();
} catch (JsonParseException e) {
e.printStackTrace();
}
Note:
The time range for scheduled messages is any time point within 864,000 seconds (10 days) starting from the current time; for example, starting from 12:00 on October 1, it can be set to 12:00 on October 11 at the most.
Scheduled messages cannot be sent in batch mode, so you need to set the enableBatch
parameter to false
when creating the producer.
Scheduled messages can be consumed only in the shared mode; otherwise, scheduling will not work (even in the key-shared mode).
Delayed message
Delayed messages are implemented by the producer's deliverAfter()
method. Below is the sample code:
String value = "message content";
long delayTime = 10L;
MessageId msgId = producer.newMessage()
.value(value.getBytes())
.deliverAfter(delayTime, TimeUnit.SECONDS)
.send();
Note:
The time range for delayed messages is 0–864,000 seconds (10 days); for example, starting from 12:00 PM on October 1, it can be set to 12:00 PM on October 11 at most.
When the delay time for messages exceeds 10 days in the Go SDK, duplicate messages may occur.
Delayed messages cannot be sent in batch mode, so you need to set the enableBatch
parameter to false
when creating the producer.
Delayed messages can be consumed only in the shared mode; otherwise, delaying will not work (even in the key-shared mode).
Use Instructions and Limits
When you use scheduled or delayed messages, we recommend you send them to a topic different from that for general messages, so you can manage them easily later while improving system stability.
When using scheduled and delayed messages, ensure that the client machine’s clock is synchronized with the server machine’s clock (set to UTC8, Beijing Time in all regions++) to prevent time discrepancies.
There is a precision deviation of about 1 second for scheduled and delayed messages.
Scheduled and delayed messages do not support batch mode (i.e., batch sending), as that mode will cause message retention. To be on the safe side, you need to set the enableBatch
parameter to false
when creating the producer.
Scheduled and delayed messages can be consumed only in the shared mode; otherwise, scheduling and delaying will not work (even in the key-shared mode).
The maximum time range for scheduled and delayed messages are both 10 days.
When using scheduled messages, you need to set a time point after the current time; otherwise, the message will be sent to the consumer immediately.
After the scheduled time point is set, the maximum message retention period (i.e., TTL) will still be calculated starting from the message sending time point; for example, if the message is scheduled to be sent in two days, but the TTL of the message is set to one day, then the message will be deleted after one day. In this case, you should make sure that the TTL is longer than the scheduled time (i.e., two days); otherwise, the message will be deleted after the TTL elapses. This is also the case for delayed messages.
In general topics, you can send scheduled and delayed messages through API as instructed in Directions and receive them.