tencent cloud

Feedback

Live Streaming Scenario

Last updated: 2024-11-22 17:19:39

    Basic knowledge

    This article mainly introduces the live broadcast function of the Video Cloud SDK. Before that, it will be helpful to understand the following basic knowledge:
    Live and on-demand The video source of live broadcast (LIVE) is pushed by the anchor in real time. Therefore, after the anchor stops pushing, the picture on the playback end will also stop immediately. And because it is a real-time live broadcast, there is no progress bar when the player plays the live URL.
    The video source of on-demand (VOD) is a video file in the cloud. As long as it is not removed from the cloud, the video can be played at any time. During playback, you can control the playback position through the progress bar. Video viewing on video websites such as Tencent Video and Youku Tudou is a typical on-demand scenario.
    Protocol support The commonly used live broadcast protocols are as follows. The App end recommends using the live broadcast address of the FLV protocol (starting with "http" and ending with ".flv"):
    Protocol
    Advantage
    Disadvantage
    Playback Latency
    HLS
    Mature, well adapted to high-concurrency scenarios
    SDK integration is required
    3–5 seconds
    FLV
    Mature, well adapted to high-concurrency scenarios
    SDK integration is required.
    2-3 seconds
    RTMP
    Relatively low latency
    Poor performance in high-concurrency scenarios
    1-3 seconds
    WebRTC
    Lowest latency
    SDK integration is required
    < 1 second

    Special Note

    Are there any restrictions? The Video Cloud SDK does not restrict the source of the playback address, that is, you can use it to play playback addresses in Tencent Cloud or non-Tencent Cloud. However, the player in the Video Cloud SDK only supports live broadcast addresses in three formats: FLV, RTMP, and HLS (m3u8), and on-demand addresses in three formats: MP4, HLS (m3u8), and FLV.
    Historical factors In the early versions of the SDK, there was only one class, TXLivePlayer, to carry the live broadcast and on-demand functions. However, as the on-demand functions became more and more, we finally separated the on-demand functions from SDK version 3.5 and handed them over to TXVodPlayer. However, in order to ensure that the compilation passes, you can still see on-demand functions such as seek in TXLivePlayer.

    Integration Guide

    Step 1: Create Player

    The TXLivePlayer module in the Video Cloud SDK is responsible for implementing the live broadcast function. The corresponding Flutter module is TXLivePlayerController.
    TXLivePlayerController controller = TXLivePlayerController();

    Step 2: Render View

    Next, we need to find a place to display the video screen of the player. The Flutter system uses Widget as the basic interface rendering unit, so you only need to prepare a Widget and adjust the layout. You can directly use TXPlayerVideo or inherit it to display, or refer to the source code to implement a custom view.
    @override
    Widget build(BuildContext context) {
    return Scaffold(
    appBar: AppBar(
    title: const Text('Live Streaming'),
    ),
    body: SafeArea(
    child: Column(
    children: [
    Container(
    height: 150,
    color: Colors.black,
    child: Center(
    child: _aspectRatio>0?AspectRatio(
    aspectRatio: _aspectRatio,
    child: TXPlayerVideo(controller: _controller),
    ):Container(),
    ),
    ),],
    ),
    ),
    );
    }

    Step 3: Start playback

    String flvUrl = "http://liteavapp.qcloud.com/live/liteavdemoplayerstreamid_demo1080p.flv";
    await _controller.startLivePlay(flvUrl);
    // If you are using a version lower than 12.0, you need to pass in playType, as shown below:
    await _controller.startLivePlay(flvUrl, playType: TXPlayType.LIVE_FLV);
    Note:
    If you integrate a version lower than 12.0 (not including 12.0), you need to pass in playType when playing the live stream.
    The meaning of playType is as follows:
    Optional value
    Enumeration value
    Meaning
    PLAY_TYPE_LIVE_RTMP
    0
    The incoming URL is the RTMP live broadcast address
    PLAY_TYPE_LIVE_FLV
    1
    The incoming URL is the FLV live broadcast address
    PLAY_TYPE_LIVE_RTMP_ACC
    5
    Low latency link address (only suitable for microphone-connected scenarios)
    PLAY_TYPE_VOD_HLS
    3
    The incoming URL is the HLS (m3u8) playback address
    Note:
    Regarding HLS (m3u8), we do not recommend using the HLS playback protocol to play live video sources on the App (although it is very suitable for on-demand playback) because the latency is too high. We recommend using the LIVE_FLV or LIVE_RTMP playback protocol on the App.

    Step 4: Pause playback

    For live broadcast, there is no real pause. The so-called live broadcast pause only freezes the picture and turns off the sound. The cloud video source is still updating, so when you call resume, it will start playing from the latest time point, which is very different from on-demand (the pause and resume of the on-demand player is the same as when playing local video files).
    // Pause
    _controller.pause();
    //
    _controller.resume();

    Step 5: End playback

    // Stop playing
    _controller.stop();

    Step 6: Message Reception

    This function can send some custom messages directly to the audience along with the audio and video lines on the streaming end. Applicable scenarios include:
    Top Conference: The streaming end sends the topic to the audience, which can achieve perfect synchronization of "audio-video-topic".
    Live Show: The streaming end sends the lyrics to the audience, and the lyrics special effects can be drawn in real time on the playback end, so it is not affected by the quality degradation of video encoding.
    Online Education: The streaming end sends the laser pen and graffiti operation to the audience, and circles and lines can be drawn in real time on the playback end.
    This function can be used through the following schemes:
    Monitor messages through onPlayerEventBroadcast, message number: PLAY_EVT_GET_MESSAGE (2012).
    _controller.onPlayerEventBroadcast.listen((event) {//Subscribe to event distribution
    if(event["event"] == 2012) {
    String msg = event["EVT_GET_MSG"];
    }
    });

    Step 7: Seamless switching of clarity

    In daily use, network conditions are constantly changing. In the case of poor network, it is best to moderately reduce the image quality to reduce lag; conversely, with a good network speed, you can watch higher quality images.
    The traditional way of switching streams is generally to replay, which will cause problems such as disconnection between the images before and after switching, black screen, and lag. Using a seamless switching solution, you can directly switch to another stream without interrupting the live broadcast.
    The clarity switch can be called at any time after the live broadcast starts. The calling method is as follows:
    // The stream currently being played is http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e.flv,
    // Now switch to a new stream with a bit rate of 900kbps
    _controller.switchStream("http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e_900.flv");
    Note:
    The seamless clarity switching function requires PTS alignment to be configured in the background. If you need it, you can submit a work order to apply for its use.

    Delay adjustment

    The live broadcast function of Tencent Cloud SDK is not based on ffmpeg for secondary development, but uses a self-developed playback engine. Therefore, compared with open source players, it has better performance in live broadcast delay control. We provide three delay adjustment modes, which are suitable for: show, game and mixed scenarios.
    The characteristics of the three modes are compared in the following table:
    Control mode
    Stuttering rate
    Average delay
    Applicable scenarios
    Principle description
    Extreme mode
    Smoother and higher
    2s - 3s
    Beauty show (top contest)
    Advantageous in delay control, suitable for scenes that are sensitive to delay size
    Smooth Mode
    Lowest lag rate
    >= 5s
    Game live broadcast (Penguin E-sports)
    Very suitable for live broadcast of games with ultra-high bitrate (such as PlayerUnknown's Battlegrounds), with the lowest lag rate
    Automatic mode
    Network adaptation
    2s - 8s
    Mixed scenarios
    The better the network on the audience side, the lower the delay; the worse the network on the audience side, the higher the delay
    The docking codes for the three modes are as follows:
    // Automatic mode
    _controller.setLiveMode(TXPlayerLiveMode.Automatic);
    // Speed ​​mode
    _controller.setLiveMode(TXPlayerLiveMode.Speed);
    // Smooth mode
    _controller.setLiveMode(TXPlayerLiveMode.Smooth);
    
    // Start playback after setting is complete
    Note:
    For more technical knowledge about lag and latency optimization, you can read Video Stutter.

    Function Usage

    The following will introduce how to use the common live broadcast function.

    1. Pause Playback

    For live broadcast, there is no real pause. The so-called live broadcast pause is just freezing the picture and turning off the sound. The cloud video source is still updating, so when you call resume, it will start playing from the latest time point, which is very different from on-demand (the pause and resume of the on-demand player is the same as when playing local video files).
    // Pause
    _controller.pause();
    // Continue
    _controller.resume();

    2. Message Reception

    This feature allows the broadcaster to send custom messages along with the audio and video stream directly to the audience. The applicable scenarios are as follows:
    Quiz Shows: The broadcaster sends the questions to the audience, achieving perfect synchronization of "audio-video-question".
    Show Live: The broadcaster sends lyrics to the audience, allowing real-time rendering of lyric effects on the playback end, unaffected by video encoding quality loss.
    Online Education: The broadcaster sends laser pen and doodle operations to the audience, allowing real-time drawing on the playback end.
    You can use this feature through the following method:
    Set the enableMessage switch in FTXLivePlayConfig to true.
    Listen for messages in TXLivePlayerController via onPlayerEventBroadcast, message ID: PLAY_EVT_GET_MESSAGE (2012).
    _controller.onPlayerEventBroadcast.listen((event) {
    if (event["event"] == TXVodPlayEvent.PLAY_EVT_GET_MESSAGE) {
    String msg = event[TXVodPlayEvent.EVT_GET_MSG];
    } else if (event["event"] == TXVodPlayEvent.PLAY_ERR_NET_DISCONNECT) {
    print("Network disconnected, streaming failed");
    }
    });

    3. Seamless Quality Switch

    During daily use, network conditions are constantly changing. In poor network conditions, it is best to appropriately reduce the video quality to reduce stuttering; conversely, if the network speed is good, you can watch higher-quality videos. Traditional stream switching usually involves restarting playback, which can cause issues like discontinuity, black screens, and stuttering. Using the seamless switching scheme allows you to switch streams directly without interrupting the live broadcast.
    Quality switches can be called at any time after the live broadcast starts. The calling method is as follows:
    // The stream currently being played is http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e.flv,
    // Now switch to the new stream with a bit rate of 900kbps
    _controller.switchStream("http://5815.liveplay.myqcloud.com/live/5815_62fe94d692ab11e791eae435c87f075e_900.flv");
    Note:
    The seamless clarity switching function requires PTS alignment to be configured in the background. If you need it, you can submit a work order to apply for its use.

    4. Latency Adjustment

    The live streaming (LVB) functionality of the Tencent Cloud SDK is not based on secondary development using ffmpeg but uses a self-developed playback engine. Therefore, compared to open-source players, it performs better in terms of live streaming latency control. We provide three latency adjustment modes suitable for different scenarios: shows, gaming, and mixed scenarios.
    Comparison of the three modes:
    Control mode
    Stuttering rate
    Average delay
    Applicable scenarios
    Principle description
    Extreme mode
    Smoother and higher
    2s - 3s
    Beauty show (top contest)
    Advantageous in delay control, suitable for scenes that are sensitive to delay size
    Smooth Mode
    Lowest lag rate
    >= 5s
    Game live broadcast (Penguin E-sports)
    Very suitable for live broadcast of games with ultra-high bitrate (such as PlayerUnknown's Battlegrounds), with the lowest lag rate
    Automatic mode
    Network adaptation
    2s - 8s
    Mixed scenarios
    The better the network on the audience side, the lower the delay; the worse the network on the audience side, the higher the delay
    Three modes of docking code
    //Automatic mode
    _controller.setCacheParams(1, 5);
    //Extreme mode
    _controller.setCacheParams(1, 1);
    //Smooth mode
    _controller.setCacheParams(5, 5);
    Note:
    For more technical knowledge about lag and latency optimization, you can read Video Stutter.

    5. Getting Video Information

    The Player SDK plays videos through a URL string, which itself does not contain video information. To obtain related information, it is necessary to load the video information by accessing the cloud server. Therefore, the SDK can only send the video information to your application via event notifications. For more content, refer to Event Listening.
    For example, you can obtain the width and height of a video through the onPlayerNetStatusBroadcast's NET_STATUS_VIDEO_WIDTH and NET_STATUS_VIDEO_HEIGHT. For specific usage, see Status Feedback (onPlayerNetStatusBroadcast).

    6. Enter Picture-in-Picture

    Currently, both platforms support Picture-in-Picture capability. On the Android side, you can pass a custom window image with a size limit of 1MB. If not provided, the default icon will be used. For IOS live streaming Picture-in-Picture, the premium version of SDK permission is required.
    _playerController.enterPictureInPictureMode(
    backIconForAndroid: backIconForAndroid,
    playIconForAndroid: playIconForAndroid,
    pauseIconForAndroid: pauseIconForAndroid,
    forwardIconForAndroid: forwardIconForAndroid);

    Listening for events

    You can monitor the TXLivePlayerController's onPlayerEventBroadcast and onPlayerNetStatusBroadcast, after which all internal status information of the SDK will be notified to you through onPlayerEventBroadcast (event notification) and onPlayerNetStatusBroadcast (status feedback).

    Event Notification (onPlayerEventBroadcast)

    1. Playback events

    Event ID
    Numerical Value
    Description
    PLAY_EVT_PLAY_BEGIN
    2004
    Video playback starts
    PLAY_EVT_PLAY_LOADING
    2007
    Video playback loading, if it can be restored, there will be a LOADING_END event afterward
    PLAY_EVT_VOD_LOADING_END
    2014
    Video playback loading ends, video continues to play
    Do not hide the playback screen after receiving PLAY_LOADING: The duration from PLAY_LOADING to PLAY_BEGIN is uncertain, it could be 5s or 5ms. Some users consider hiding the screen during LOADING and displaying it at BEGIN, which causes severe screen flickering (especially in live streaming scenarios).The recommended practice is to overlay a semi-transparent loading animation on the video playback screen.

    2. Stop events

    Event ID
    Numerical Value
    Description
    PLAY_EVT_PLAY_END
    2006
    Video playback ends
    PLAY_ERR_NET_DISCONNECT
    -2301
    Network disconnected, and after multiple reconnections fails to restore, for more retries please restart playback manually
    PLAY_ERR_HLS_KEY
    -2305
    Failed to get HLS decryption key
    How can one determine if a live stream has ended? Due to different implementation principles based on various standards, many live streams usually do not emit an 'end event' (2006). In such cases, the expected behavior is: after the broadcaster stops streaming, the SDK will quickly detect a failure in data stream retrieval (PLAY_WARNING_RECONNECT), then begin to retry until after three failed attempts, it will then emit a PLAY_ERR_NET_DISCONNECT event.
    Therefore, both error codes 2006 and -2301 need to be listened for and used to determine the end of live streaming.

    3. Warning events

    You can ignore the following events, they are used to inform you about some internal events of the SDK.
    Event ID
    Numerical Value
    Description
    PLAY_WARNING_VIDEO_DECODE_FAIL
    2101
    Failed to decode the current video frame
    PLAY_WARNING_AUDIO_DECODE_FAIL
    2102
    Failed to decode the current audio frame
    PLAY_WARNING_RECONNECT
    2103
    Network Disconnection, automatic reconnection initiated (if reconnection fails more than three times, it will directly throw the PLAY_ERR_NET_DISCONNECT)
    PLAY_WARNING_HW_ACCELERATION_FAIL
    2106
    Failed to enable the hardware decoder, and the software decoder was used

    Status feedback (onPlayerNetStatusBroadcast)

    The notification is triggered once every second to provide real-time feedback on the current status of the pusher. It can act as a dashboard to inform you of what is happening inside the SDK so you can better understand the current network conditions and video information.
    Evaluation parameters
    Description
    NET_STATUS_CPU_USAGE
    Current instantaneous CPU utilization
    NET_STATUS_VIDEO_WIDTH
    Video resolution - width
    NET_STATUS_VIDEO_HEIGHT
    Video resolution - height
    NET_STATUS_NET_SPEED
    Current network data reception speed
    NET_STATUS_VIDEO_FPS
    Current video frame rate of streaming media
    NET_STATUS_VIDEO_BITRATE
    Current video bitrate in Kbps of streaming media
    NET_STATUS_AUDIO_BITRATE
    Current audio bitrate in Kbps of streaming media
    NET_STATUS_CACHE_SIZE
    Jitterbuffer size; if the current length of the buffer zone is 0, it indicates that lag is imminent
    NET_STATUS_SERVER_IP
    Connected server IP
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support