Real-time synchronization of song progress is required in the real-time solution to avoid increasing end-to-end delay due to song errors after the start of the performance. Synchronizing the song requires using NTP time. The local clocks of different devices are not consistent and there is a certain error, so Tencent Cloud's self-developed NTP service needs to be introduced. At the same time, users who join the chorus midway also need to synchronize the song progress, and only after synchronizing the progress can they participate in the chorus.
Implementation process
The method of synchronizing songs is that the main singer user agrees to start playing the song at a future point in time (such as N seconds after delay), and other users participate in the chorus. The time of each end is based on NTP time, which will start synchronizing after the TRTC SDK is initialized.
The specific process is as follows:
1. Each end performs NTP time calibration, updates and obtains the latest NTP time T to the TRTC cloud.
2. The main singer end sends a chorus signal (custom message) to agree on the start time T2 of the chorus.
3. Preload the song locally based on T2 and play it at a scheduled time.
4. Other chorus users execute step 3 after receiving the chorus signal.
5. During the process, the local song playback progress is verified, and seek calibration is performed when the difference between TE and TC exceeds 50ms.
Note:
The 50ms error here is a typical value, which can be adjusted appropriately according to the business tolerance. It is recommended to fluctuate around 50ms.
Timing diagram
The song synchronization timing can mainly be divided into three parts: NTP time calibration, sending and receiving chorus signals, and correcting the song playback progress. The following will provide specific code implementation for these three parts.
Key code implementation
1. NTP calibration time service
[TXLiveBase updateNetworkTime];
- (void)onUpdateNetworkTime:(int)errCode message:(NSString *)errMsg {
if (errCode == 0) {
NSInteger ntpTime = [TXLiveBase getNetworkTimestamp];
} else {
[TXLiveBase updateNetworkTime];
}
}
2. Sending chorus signals on the main singer end
NSDictionary *json = @{
@"cmd": @"startChorus",
@"startPlayMusicTS": @(startTs),
@"musicId": @"musicId",
@"musicDuration": @(musicDuration),
};
NSString *jsonString = [self jsonStringFrom:json];
[trtcCloud sendCustomMessage:jsonString reliable:NO];
Note:
It is recommended that the main singer send chorus signal messages to the room at a fixed time frequency in a loop, so that chorus users can join in midway;
The reason for not using SEI messages to send chorus signals is that the SEI information will be inserted into the video frame, causing a lot of invalid information to be carried in the video stream pulled by the audience side.
3. Receiving chorus signals on the chorus end
- (void)onRecvCustomCmdMsgUserId:(NSString *)userId cmdID:(NSInteger)cmdId seq:(UInt32)seq message:(NSData *)message {
NSString *msg = [[NSString alloc] initWithData:message encoding:NSUTF8StringEncoding];
NSError *error;
NSDictionary *json = [NSJSONSerialization JSONObjectWithData:[msg dataUsingEncoding:NSUTF8StringEncoding]
options:NSJSONReadingMutableContainers
error:&error];
NSObject *cmdObj = [json objectForKey:@"cmd"];
NSInteger musicDuration = [[json objectForKey:@"musicDuration"] integerValue];
NSString *cmd = (NSString *)cmdObj;
if ([cmd isEqualToString:@"startChorus"]) {
NSObject *startPlayMusicTsObj = [json objectForKey:@"startPlayMusicTS"];
NSString *musicId = [json objectForKey:@"musicId"];
NSInteger startPlayMusicTs = ((NSNumber *)startPlayMusicTsObj).longLongValue;
NSInteger startDelayMS = labs(startPlayMusicTs - [TXLiveBase getNetworkTimestamp]);
NSDictionary *jsonDict = @{
@"api": @"preloadMusic",
@"params": @{
@"musicId": @(musicId),
@"path": path,
@"startTimeMS": @(startDelayMS),
}
};
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:jsonDict options:0 error:NULL];
NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
[subCloud callExperimentalAPI:jsonString];
TXAudioMusicParam *param = [[TXAudioMusicParam alloc] init];
param.ID = musicId;
param.path = url;
param.loopCount = 0;
param.publish = NO;
[[subCloud getAudioEffectManager] startPlayMusic:param onStart:^(NSInteger errCode) {
} onProgress:^(NSInteger progressMs, NSInteger durationMs) {
} onComplete:^(NSInteger errCode) {
}];
}
}
Note:
After the chorus end receives the first startChorus signal, the status should be changed from "not singing" to "singing", and no longer respond to the startChorus signal to avoid restarting the BGM playback.
4. Song playback progress correction
self.startPlayChorusMusicTs;
NSInteger currentProgress = [[self audioEffecManager] getMusicCurrentPosInMS:self.currentPlayMusicID];
NSInteger estimatedProgress = [TXLiveBase getNetworkTimestamp] - self.startPlayChorusMusicTs;
if (estimatedProgress >= 0 && labs(currentProgress - estimatedProgress) > 50) {
[[subCloud getAudioEffectManager] seekMusicToPosInMS:self.currentPlayMusicID pts:estimatedProgress];
}
Was this page helpful?