tencent cloud

Feedback

TUIMediaMixingManager

Last updated: 2024-09-10 15:30:52

    Introduction

    TUIMediaMixingManager is a local multimedia mixing manager that supports mixing multiple local video streams into one video stream for streaming.
    Installation
    // npm
    npm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save
    
    // pnpm
    pnpm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save
    
    // yarn
    yarn add @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3
    
    Example code:
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();

    API

    Method
    Description
    Set the program path for the local mixing service process
    Set the local mixing video preview window and area
    Add a multimedia source
    Remove a multimedia source
    Update a multimedia source
    Set camera capture parameters
    Start local mixing and streaming
    Stop local mixing and streaming
    Modify local mixing parameters

    Type Definitions

    Type
    Description
    Camera capture parameters
    Local mixing configuration parameters
    Media source data
    TUIRect
    Rectangular coordinate area, including left, top, right, bottom properties
    Screen/window data
    Mixed video encoding parameters

    Enumerations

    Enum
    Description
    Camera capture mode
    Media source fill mode
    Media source mirror type
    Media source rotation angle
    Media source type
    Resolution mode (landscape resolution/vertical resolution)
    Video resolution

    API explaination

    setMediaServerPath()

    Set the program path for the local mixing service process. After creating an instance object, this method must be called first for other methods to take effect.
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    
    async function initMediaServer() {
    const isPackaged = location.href.startsWith("file"); // Same with Electron `app.isPackaged`
    const resourcesPath = (globalThis as any).process.resourcesPath;
    let mediaServerPath = '';
    if (isPackaged) {
    mediaServerPath = `${resourcesPath}\\\\liteav_media_server.exe`;
    } else {
    const appPath = ''; // Value from Electron main process: app.getAppPath()
    mediaServerPath = `${appPath}\\\\node_modules\\\\trtc-electron-sdk\\\\build\\\\Release\\\\liteav_media_server.exe`;
    }
    mediaMixingManager.setMediaServerPath(mediaServerPath);
    }
    
    initMediaServer();
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    path
    string
    -
    -
    Program path for the local mixing service process
    Return Promise<void>

    setDisplayParams()

    Set the local mixing video preview window and area
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    
    // Value from Electron main BrowserWindow.getNativeWindowHandle()
    const nativeWindowHandle = new Uint8Array(8);
    
    const previewDOMNode = document.getElementByID("preview-media-mixing");
    const clientRect = previewDOMNode.getBoundingClientRect();
    mediaMixingManager.setDisplayParams(window.nativeWindowHandle, {
    left: clientRect.left * window.devicePixelRatio, // Notice: window.devicePixelRatio
    right: clientRect.right * window.devicePixelRatio, // Notice: window.devicePixelRatio
    top: clientRect.top * window.devicePixelRatio, // Notice: window.devicePixelRatio
    bottom: clientRect.bottom * window.devicePixelRatio, // Notice: window.devicePixelRatio
    });
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    windowID
    Uint8Array
    -
    -
    Mixed video preview window ID, obtained through Electron API BrowserView.getNativeWindowHandle()
    rect
    TUIRect
    -
    -
    Mixed video preview area position and size
    Return void

    addMediaSource()

    Add media source
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    const mediaSource:TUIMediaSource = {
    sourceType: TUIMediaSourceType.kCamera,
    sourceId: '', // camera ID, screen ID, window ID or image path
    zOrder: 1, // should be unique among all media sources
    rect: {
    left: 0,
    top: 0,
    right: 640,
    bottom: 360
    },
    };
    mediaMixingManager.addMediaSource(mediaSource);
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    mediaSource
    -
    -
    Media source information
    Return Promise<TUIRect> Default position in mixing video image

    removeMediaSource()

    Remove media source
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    const mediaSource:TUIMediaSource = {
    sourceType: TUIMediaSourceType.kCamera, // required
    sourceId: '', // required
    zOrder: 1, // any valid value
    rect: { // any valid value
    left: 0,
    top: 0,
    right: 640,
    bottom: 360
    },
    };
    mediaMixingManager.removeMediaSource(mediaSource);
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    mediaSource
    -
    -
    Media source information
    Return Promise<void>

    updateMediaSource()

    Update media source
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    const mediaSource:TUIMediaSource = {
    sourceType: TUIMediaSourceType.kCamera, // required
    sourceId: '', // required
    zOrder: 1, // should be unique among all media sources
    rect: { // any valid value
    left: 0,
    top: 0,
    right: 640,
    bottom: 360
    },
    };
    mediaMixingManager.updateMediaSource(mediaSource);
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    mediaSource
    -
    -
    Media source information
    Return Promise<void>

    setCameraCaptureParam()

    Set camera capture parameter
    import TUIRoomEngine, { TUIMediaMixingManager, TUICameraCaptureMode } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    mediaMixingManager.setCameraCaptureParam('<camera ID>', {
    mode: TUICameraCaptureMode.kCameraCaptureManual,
    width: 1280,
    height: 720,
    });
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    cameraId
    string
    Non-empty
    -
    Camera ID
    param
    -
    -
    Camera capture parameters
    Return

    startPublish()

    Start local mixing and streaming Note: Streaming can only be successful after entering the room.
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    mediaMixingManager.startPublish();
    Return Promise<void>

    stopPublish()

    Stop local streaming, but still mixing that can be previewed.
    import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    mediaMixingManager.stopPublish();
    Return Promise<void>

    updatePublishParams()

    Modify local mixing parameters
    import TUIRoomEngine, {
    TUIMediaMixingManager,
    TUIResolutionMode,
    TUIVideoResolution
    } from '@tencentcloud/tuiroom-engine-electron';
    
    const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
    mediaMixingManager.updatePublishParams({
    inputSourceList: [], // TUIMediaSource[], can be empty
    videoEncoderParams: {
    videoResolution: TUIVideoResolution.kVideoResolution_1920_1080,
    resMode: TUIResolutionMode.kResolutionMode_Landscape,
    videoFps: 30,
    videoBitrate: 3000,
    minVideoBitrate: 2000,
    enableAdjustRes: true
    },
    canvasColor: 0xc6c6c6, // integer number of RGB
    });
    Parameters
    Parameter
    Type
    Validation
    Default value
    Description
    params
    -
    -
    Local mixing configuration parameters
    Return Promise<void>

    Type explaination

    TUICameraCaptureParam

    Camera capture parameters
    Enumeration item
    Type
    Description
    mode
    Camera capture mode
    width
    number
    Captured image width
    height
    number
    Captured image height

    TUILocalMediaTranscodingParams

    Local medix mixing transcoding configuration parameters
    Field
    Type
    Description
    inputSourceList
    Media source list
    videoEncoderParams
    Mixed-stream video encoding parameters
    canvasColor
    number
    Mixed-stream video background color, RGB format, default: 0x0

    TUIMediaSource

    Media source data
    Field
    Type
    Description
    sourceType
    Media source type
    sourceId
    string
    Unique ID of media source
    rect
    TUIRect
    Coordinate position relative to the local mixing layout
    zOrder
    number
    Display hierarchy. Media sources with higher display hierarchy will cover those with lower display hierarchy, and multiple media sources cannot have the same hierarchy.
    rotation
    Optional, rotation angle
    fillMode
    Optional, fill mode
    mirrorType
    Optional, mirror mode
    isSelected
    boolean
    Optional, default: false, whether the media source is selected

    TUIRect

    Rectangular coordinate area
    Field
    Type
    Description
    left
    number
    Left line coordinate
    top
    number
    Top line coordinate
    right
    number
    Right line coordinate
    bottom
    number
    Bottom line coordinate

    TUIScreenCaptureSourceInfo

    Screen/window data
    Field
    Type
    Description
    type
    Media source type: screen/window
    sourceId
    number
    Unique ID
    sourceName
    string
    Name
    isMinimizeWindow
    boolean
    Whether the window is minimized
    isMainScreen
    boolean
    Whether it is the main screen
    rect
    TUIRect
    Relative coordinate position on the display screen

    TUIVideoEncParam

    Mixed-stream video encoding parameters
    Field
    Type
    Description
    videoResolution
    Video resolution
    resMode
    Video portrait/landscape mode
    videoFps
    number
    Frame rate
    videoBitrate
    number
    Bitrate
    minVideoBitrate
    number
    Minimum bitrate
    enableAdjustRes
    boolean
    Whether to enable adaptive bitrate adjustment, default: true

    Enumerations explaination

    TUICameraCaptureMode

    Camera capture mode
    Enumeration item
    Type
    Description
    kCameraResolutionStrategyAuto
    number
    Automatically adjust capture parameters. The SDK selects appropriate camera output parameters based on the actual performance of the capture device and network conditions, maintaining a balance between device performance and video preview quality.
    kCameraResolutionStrategyPerformance
    number
    Priority on device performance. The SDK selects the closest camera output parameters based on the user's settings for encoder resolution and frame rate, thereby ensuring device performance.
    kCameraResolutionStrategyHighQuality
    number
    Priority on video preview quality. The SDK selects higher camera output parameters to improve the quality of the preview video. In this case, more CPU and memory will be consumed for video preprocessing.
    kCameraCaptureManual
    number
    Allows users to set the local camera capture video width and height.

    TUIMediaFillMode

    Media displaying Fill mode
    Enumeration item
    Type
    Description
    kMediaFillMode_Fill
    number
    The media fills the display area, and the parts of the video that exceed the display window will be cut off. The picture content may not be displayed completely
    kMediaFillMode_Fit
    number
    The long side of the media fills the display area, and the short side area will be filled with black. The picture content is displayed completely

    TUIMediaMirrorType

    Mirror mode
    Enumeration item
    Type
    Description
    kMediaMirrorType_Auto
    number
    Automatic mode. During local preview, the front camera is mirrored, and the rear camera is not mirrored
    kMediaMirrorType_Enable
    number
    Enable mirror
    kMediaMirrorType_Disable
    number
    Disable mirror

    TUIMediaRotation

    Rotation angle
    Enumeration item
    Type
    Description
    kMediaRotation0
    number
    Rotate 0 degrees
    kMediaRotation90
    number
    Rotate 90 degrees
    kMediaRotation180
    number
    Rotate 180 degrees
    kMediaRotation270
    number
    Rotate 270 degrees

    TUIMediaSourceType

    Media source type
    Enumeration item
    Type
    Description
    kCamera
    number
    Camera
    kScreen
    number
    Screen/Window
    kImage
    number
    Image

    TUIResolutionMode

    Video resolution mode
    Enumeration item
    Type
    Description
    kResolutionMode_Landscape
    number
    Landscape mode
    kResolutionMode_Portrait
    number
    Portrait mode

    TUIVideoResolution

    Video resolution. Only landscape resolutions are defined here. If you want to use a portrait resolution such as 360 × 640, you need to specify TUIResolutionMode as Portrait at the same time.
    Enumeration item
    Type
    Description
    kVideoResolution_120_120
    number
    Recommended bitrate (VideoCall) 80kbps; Recommended bitrate (LIVE) 120kbps
    kVideoResolution_160_160
    number
    Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps
    kVideoResolution_270_270
    number
    Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
    kVideoResolution_480_480
    number
    Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 500kbps
    kVideoResolution_160_120
    number
    Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps
    kVideoResolution_240_180
    number
    Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps
    kVideoResolution_280_210
    number
    Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
    kVideoResolution_320_240
    number
    Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 375kbps
    kVideoResolution_400_300
    number
    Recommended bitrate (VideoCall) 300kbps; Recommended bitrate (LIVE) 450kbps
    kVideoResolution_480_360
    number
    Recommended bitrate (VideoCall) 400kbps; Recommended bitrate (LIVE) 600kbps
    kVideoResolution_640_480
    number
    Recommended bitrate (VideoCall) 600kbps; Recommended bitrate (LIVE) 900kbps
    kVideoResolution_960_720
    number
    Recommended bitrate (VideoCall) 1000kbps; Recommended bitrate (LIVE) 1500kbps
    kVideoResolution_160_90
    number
    Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps
    kVideoResolution_256_144
    number
    Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
    kVideoResolution_320_180
    number
    Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 400kbps
    kVideoResolution_480_270
    number
    Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 550kbps
    kVideoResolution_640_360
    number
    Recommended bitrate (VideoCall) 500kbps; Recommended bitrate (LIVE) 900kbps
    kVideoResolution_960_540
    number
    Recommended bitrate (VideoCall) 850kbps; Recommended bitrate (LIVE) 1300kbps
    kVideoResolution_1280_720
    number
    Recommended bitrate (VideoCall) 1200kbps; Recommended bitrate (LIVE) 1800kbps
    kVideoResolution_1920_1080
    number
    Recommended bitrate (VideoCall) 2000kbps; Recommended bitrate (LIVE) 3000kbpy
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support