Basic knowledge
This article mainly introduces the live broadcast function of the Video Cloud SDK. Before that, it will be helpful to understand the following basic knowledge:
Live and on-demand
The video source of live broadcast (LIVE) is pushed by the anchor in real time. Therefore, after the anchor stops pushing, the picture on the playback end will also stop immediately. And because it is a real-time live broadcast, there is no progress bar when the player plays the live URL.
The video source of on-demand (VOD) is a video file in the cloud. As long as it is not removed from the cloud, the video can be played at any time. During playback, you can control the playback position through the progress bar. Video viewing on video websites such as Tencent Video and Youku Tudou is a typical on-demand scenario.
Protocol support
The commonly used live broadcast protocols are as follows. The App end recommends using the live broadcast address of the FLV protocol (starting with "http" and ending with ".flv"):
|
HLS | Mature, well adapted to high-concurrency scenarios | SDK integration is required | 3–5 seconds |
FLV | Mature, well adapted to high-concurrency scenarios | SDK integration is required. | 2-3 seconds |
RTMP | Relatively low latency | Poor performance in high-concurrency scenarios | 1-3 seconds |
WebRTC | Lowest latency | SDK integration is required | < 1 second |
Special Note
Are there any restrictions?
The Video Cloud SDK does not restrict the source of the playback address, that is, you can use it to play playback addresses in Tencent Cloud or non-Tencent Cloud. However, the player in the Video Cloud SDK only supports live broadcast addresses in three formats: FLV, RTMP, and HLS (m3u8), and on-demand addresses in three formats: MP4, HLS (m3u8), and FLV.
Historical factors
In the early versions of the SDK, there was only one class, TXLivePlayer, to carry the live broadcast and on-demand functions. However, as the on-demand functions became more and more, we finally separated the on-demand functions from SDK version 3.5 and handed them over to TXVodPlayer. However, in order to ensure that the compilation passes, you can still see on-demand functions such as seek in TXLivePlayer.
Integration Guide
Step 1: Create Player
The TXLivePlayer module in the Video Cloud SDK is responsible for implementing the live broadcast function. The corresponding Flutter module is TXLivePlayerController.
TXLivePlayerController controller = TXLivePlayerController();
Step 2: Render View
Next, we need to find a place to display the video screen of the player. The Flutter system uses Widget as the basic interface rendering unit, so you only need to prepare a Widget and adjust the layout. You can directly use TXPlayerVideo or inherit it to display, or refer to the source code to implement a custom view.
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('Live Streaming'),
),
body: SafeArea(
child: Column(
children: [
Container(
height: 150,
color: Colors.black,
child: Center(
child: _aspectRatio>0?AspectRatio(
aspectRatio: _aspectRatio,
child: TXPlayerVideo(controller: _controller),
):Container(),
),
),],
),
),
);
}
Step 3: Start playback
String flvUrl = "http://liteavapp.qcloud.com/live/liteavdemoplayerstreamid_demo1080p.flv";
await _controller.startLivePlay(flvUrl);
await _controller.startLivePlay(flvUrl, playType: TXPlayType.LIVE_FLV);
Note:
If you integrate a version lower than 12.0 (not including 12.0), you need to pass in playType when playing the live stream.
The meaning of playType is as follows:
|
PLAY_TYPE_LIVE_RTMP | 0 | The incoming URL is the RTMP live broadcast address |
PLAY_TYPE_LIVE_FLV | 1 | The incoming URL is the FLV live broadcast address |
PLAY_TYPE_LIVE_RTMP_ACC | 5 | Low latency link address (only suitable for microphone-connected scenarios) |
PLAY_TYPE_VOD_HLS | 3 | The incoming URL is the HLS (m3u8) playback address |
Note:
Regarding HLS (m3u8), we do not recommend using the HLS playback protocol to play live video sources on the App (although it is very suitable for on-demand playback) because the latency is too high. We recommend using the LIVE_FLV or LIVE_RTMP playback protocol on the App.
Step 4: Pause playback
For live broadcast, there is no real pause. The so-called live broadcast pause only freezes the picture and turns off the sound. The cloud video source is still updating, so when you call resume, it will start playing from the latest time point, which is very different from on-demand (the pause and resume of the on-demand player is the same as when playing local video files).
// Pause
_controller.pause();
//
_controller.resume();
Step 5: End playback
// Stop playing
_controller.stop();
Step 6: Message Reception
This function can send some custom messages directly to the audience along with the audio and video lines on the streaming end. Applicable scenarios include:
Top Conference: The streaming end sends the topic to the audience, which can achieve perfect synchronization of "audio-video-topic".
Live Show: The streaming end sends the lyrics to the audience, and the lyrics special effects can be drawn in real time on the playback end, so it is not affected by the quality degradation of video encoding.
Online Education: The streaming end sends the laser pen and graffiti operation to the audience, and circles and lines can be drawn in real time on the playback end.
This function can be used through the following schemes:
Monitor messages through onPlayerEventBroadcast, message number: PLAY_EVT_GET_MESSAGE (2012).
_controller.onPlayerEventBroadcast.listen((event) {//Subscribe to event distribution
if(event["event"] == 2012) {
String msg = event["EVT_GET_MSG"];
}
});
Step 7: Seamless switching of clarity
In daily use, network conditions are constantly changing. In the case of poor network, it is best to moderately reduce the image quality to reduce lag; conversely, with a good network speed, you can watch higher quality images.
The traditional way of switching streams is generally to replay, which will cause problems such as disconnection between the images before and after switching, black screen, and lag. Using a seamless switching solution, you can directly switch to another stream without interrupting the live broadcast.
The clarity switch can be called at any time after the live broadcast starts. The calling method is as follows:
_controller.switchStream("http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e_900.flv");
Note:
The seamless clarity switching function requires PTS alignment to be configured in the background. If you need it, you can submit a work order to apply for its use. Delay adjustment
The live broadcast function of Tencent Cloud SDK is not based on ffmpeg for secondary development, but uses a self-developed playback engine. Therefore, compared with open source players, it has better performance in live broadcast delay control. We provide three delay adjustment modes, which are suitable for: show, game and mixed scenarios.
The characteristics of the three modes are compared in the following table:
|
Extreme mode | Smoother and higher | 2s - 3s | Beauty show (top contest) | Advantageous in delay control, suitable for scenes that are sensitive to delay size |
Smooth Mode | Lowest lag rate | >= 5s | Game live broadcast (Penguin E-sports) | Very suitable for live broadcast of games with ultra-high bitrate (such as PlayerUnknown's Battlegrounds), with the lowest lag rate |
Automatic mode | Network adaptation | 2s - 8s | Mixed scenarios | The better the network on the audience side, the lower the delay; the worse the network on the audience side, the higher the delay |
The docking codes for the three modes are as follows:
// Automatic mode
_controller.setLiveMode(TXPlayerLiveMode.Automatic);
// Speed mode
_controller.setLiveMode(TXPlayerLiveMode.Speed);
// Smooth mode
_controller.setLiveMode(TXPlayerLiveMode.Smooth);
// Start playback after setting is complete
Note:
For more technical knowledge about lag and latency optimization, you can read Video Stutter. Function Usage
The following will introduce how to use the common live broadcast function.
1. Pause Playback
For live broadcast, there is no real pause. The so-called live broadcast pause is just freezing the picture and turning off the sound. The cloud video source is still updating, so when you call resume, it will start playing from the latest time point, which is very different from on-demand (the pause and resume of the on-demand player is the same as when playing local video files).
// Pause
_controller.pause();
// Continue
_controller.resume();
2. Message Reception
This feature allows the broadcaster to send custom messages along with the audio and video stream directly to the audience. The applicable scenarios are as follows:
Quiz Shows: The broadcaster sends the questions to the audience, achieving perfect synchronization of "audio-video-question".
Show Live: The broadcaster sends lyrics to the audience, allowing real-time rendering of lyric effects on the playback end, unaffected by video encoding quality loss.
Online Education: The broadcaster sends laser pen and doodle operations to the audience, allowing real-time drawing on the playback end.
You can use this feature through the following method:
Set the enableMessage
switch in FTXLivePlayConfig
to true
.
Listen for messages in TXLivePlayerController
via onPlayerEventBroadcast
, message ID: PLAY_EVT_GET_MESSAGE (2012).
_controller.onPlayerEventBroadcast.listen((event) {
if (event["event"] == TXVodPlayEvent.PLAY_EVT_GET_MESSAGE) {
String msg = event[TXVodPlayEvent.EVT_GET_MSG];
} else if (event["event"] == TXVodPlayEvent.PLAY_ERR_NET_DISCONNECT) {
print("Network disconnected, streaming failed");
}
});
3. Seamless Quality Switch
During daily use, network conditions are constantly changing. In poor network conditions, it is best to appropriately reduce the video quality to reduce stuttering; conversely, if the network speed is good, you can watch higher-quality videos. Traditional stream switching usually involves restarting playback, which can cause issues like discontinuity, black screens, and stuttering. Using the seamless switching scheme allows you to switch streams directly without interrupting the live broadcast.
Quality switches can be called at any time after the live broadcast starts. The calling method is as follows:
// The stream currently being played is http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e.flv,
// Now switch to the new stream with a bit rate of 900kbps
_controller.switchStream("http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e_900.flv");
Note:
The seamless clarity switching function requires PTS alignment to be configured in the background. If you need it, you can submit a work order to apply for its use. 4. Latency Adjustment
The live streaming (LVB) functionality of the Tencent Cloud SDK is not based on secondary development using ffmpeg but uses a self-developed playback engine. Therefore, compared to open-source players, it performs better in terms of live streaming latency control. We provide three latency adjustment modes suitable for different scenarios: shows, gaming, and mixed scenarios.
Comparison of the three modes:
|
Extreme mode | Smoother and higher | 2s - 3s | Beauty show (top contest) | Advantageous in delay control, suitable for scenes that are sensitive to delay size |
Smooth Mode | Lowest lag rate | >= 5s | Game live broadcast (Penguin E-sports) | Very suitable for live broadcast of games with ultra-high bitrate (such as PlayerUnknown's Battlegrounds), with the lowest lag rate |
Automatic mode | Network adaptation | 2s - 8s | Mixed scenarios | The better the network on the audience side, the lower the delay; the worse the network on the audience side, the higher the delay |
Three modes of docking code
_controller.setCacheParams(1, 5);
_controller.setCacheParams(1, 1);
_controller.setCacheParams(5, 5);
Note:
For more technical knowledge about lag and latency optimization, you can read Video Stutter. 5. Getting Video Information
The Player SDK plays videos through a URL string, which itself does not contain video information. To obtain related information, it is necessary to load the video information by accessing the cloud server. Therefore, the SDK can only send the video information to your application via event notifications. For more content, refer to Event Listening. For example, you can obtain the width and height of a video through the onPlayerNetStatusBroadcast
's NET_STATUS_VIDEO_WIDTH
and NET_STATUS_VIDEO_HEIGHT
. For specific usage, see Status Feedback (onPlayerNetStatusBroadcast). 6. Enter Picture-in-Picture
Currently, both platforms support Picture-in-Picture capability. On the Android side, you can pass a custom window image with a size limit of 1MB. If not provided, the default icon will be used. For IOS live streaming Picture-in-Picture, the premium version of SDK permission is required.
_playerController.enterPictureInPictureMode(
backIconForAndroid: backIconForAndroid,
playIconForAndroid: playIconForAndroid,
pauseIconForAndroid: pauseIconForAndroid,
forwardIconForAndroid: forwardIconForAndroid);
Listening for events
You can monitor the TXLivePlayerController
's onPlayerEventBroadcast
and onPlayerNetStatusBroadcast
, after which all internal status information of the SDK will be notified to you through onPlayerEventBroadcast
(event notification) and onPlayerNetStatusBroadcast
(status feedback).
Event Notification (onPlayerEventBroadcast)
1. Playback events
|
PLAY_EVT_PLAY_BEGIN | 2004 | Video playback starts |
PLAY_EVT_PLAY_LOADING | 2007 | Video playback loading, if it can be restored, there will be a LOADING_END event afterward |
PLAY_EVT_VOD_LOADING_END | 2014 | Video playback loading ends, video continues to play |
Do not hide the playback screen after receiving PLAY_LOADING: The duration from PLAY_LOADING to PLAY_BEGIN is uncertain, it could be 5s or 5ms. Some users consider hiding the screen during LOADING and displaying it at BEGIN, which causes severe screen flickering (especially in live streaming scenarios).The recommended practice is to overlay a semi-transparent loading animation on the video playback screen.
2. Stop events
|
PLAY_EVT_PLAY_END | 2006 | Video playback ends |
PLAY_ERR_NET_DISCONNECT | -2301 | Network disconnected, and after multiple reconnections fails to restore, for more retries please restart playback manually |
PLAY_ERR_HLS_KEY | -2305 | Failed to get HLS decryption key |
How can one determine if a live stream has ended?
Due to different implementation principles based on various standards, many live streams usually do not emit an 'end event' (2006). In such cases, the expected behavior is: after the broadcaster stops streaming, the SDK will quickly detect a failure in data stream retrieval (PLAY_WARNING_RECONNECT), then begin to retry until after three failed attempts, it will then emit a PLAY_ERR_NET_DISCONNECT event.
Therefore, both error codes 2006 and -2301 need to be listened for and used to determine the end of live streaming.
3. Warning events
You can ignore the following events, they are used to inform you about some internal events of the SDK.
|
PLAY_WARNING_VIDEO_DECODE_FAIL | 2101 | Failed to decode the current video frame |
PLAY_WARNING_AUDIO_DECODE_FAIL | 2102 | Failed to decode the current audio frame |
PLAY_WARNING_RECONNECT | 2103 | Network Disconnection, automatic reconnection initiated (if reconnection fails more than three times, it will directly throw the PLAY_ERR_NET_DISCONNECT) |
PLAY_WARNING_HW_ACCELERATION_FAIL | 2106 | Failed to enable the hardware decoder, and the software decoder was used |
Status feedback (onPlayerNetStatusBroadcast)
The notification is triggered once every second to provide real-time feedback on the current status of the pusher. It can act as a dashboard to inform you of what is happening inside the SDK so you can better understand the current network conditions and video information.
|
NET_STATUS_CPU_USAGE | Current instantaneous CPU utilization |
NET_STATUS_VIDEO_WIDTH | Video resolution - width |
NET_STATUS_VIDEO_HEIGHT | Video resolution - height |
NET_STATUS_NET_SPEED | Current network data reception speed |
NET_STATUS_VIDEO_FPS | Current video frame rate of streaming media |
NET_STATUS_VIDEO_BITRATE | Current video bitrate in Kbps of streaming media |
NET_STATUS_AUDIO_BITRATE | Current audio bitrate in Kbps of streaming media |
NET_STATUS_CACHE_SIZE | Jitterbuffer size; if the current length of the buffer zone is 0, it indicates that lag is imminent |
NET_STATUS_SERVER_IP | Connected server IP |
Was this page helpful?