tencent cloud

Feedback

Last updated: 2024-07-03 16:15:44

    Function Description

    This document will describe how to implement the virtual background feature during a call.
    Original Camera
    Background Blur
    Background Image
    
    
    
    
    
    
    
    
    

    Prerequisites

    The SdkAppId that requires the use of the virtual background feature has subscribed to the RTC-Engine Monthly Packages, Pro edition (1499 USD/month).
    TRTC SDK version must be ≥ 5.5.0
    System and configuration requirements for various platforms are as follows:
    Platform
    Operating System
    Browser Version
    FPS
    Recommended Configuration
    Remarks
    Web
    Windows
    Chrome 90+Firefox 90+Edge 97+
    30
    Memory: 16GBCPU: i5-10500GPU: Dedicated 2GB
    It is recommended to use the latest version of Chrome browser (Enable hardware acceleration in the browser)
    15
    Memory: 8GBCPU: i3-8300GPU: Integrated Intel 1GB
    Mac
    Chrome 98+Firefox 96+Safari 14+
    30
    2019 MacBookMemory: 16GB (2667MHz)CPU: i7 (6-core 2.60GHz)GPU: AMD Radeon 5300M
    Android
    ChromeFirefox Browser
    30
    High-end devices (e.g., Qualcomm Snapdragon 8 Gen1)
    It is recommended to use Chrome, or Firefox browsers and other mainstream browsers
    20
    Mid-range devices (e.g., MediaTek Dimensity 8000-MAX)
    10
    Low-end devices (e.g., Qualcomm Snapdragon 660)
    iOS
    ChromeSafariFirefox
    30
    iPhone 13
    - Requires iOS 14.4 or above- It is recommended to use Chrome or Safari browsers
    20
    iPhone XR

    Implementation Steps

    Registering the Plugin

    import { VirtualBackground } from 'trtc-sdk-v5/plugins/video-effect/virtual-background';
    
    let trtc = TRTC.create({ plugins: [VirtualBackground] });

    Starting the Local Camera

    await trtc.startLocalVideo();

    Enabling the Virtual Background Plugin

    await trtc.startPlugin('VirtualBackground', {
    sdkAppId: 123123,
    userId: 'userId_123',
    userSig: 'your_userSig'
    });
    Note:
    The first time you enable this feature, it requires loading computing resources, which has been automatically processed for you. For a more ultimate startup experience, please refer to the end of the document: FAQ > How to speed up startup?

    Updating Parameters as Needed

    // Changing to an image background
    await trtc.updatePlugin('VirtualBackground', {
    type: 'image',
    src: 'https://picsum.photos/seed/picsum/200/300'
    });

    Disabling Virtual Background

    await trtc.stopPlugin('VirtualBackground');

    API Documentation

    trtc.startPlugin('VirtualBackground', options)

    Used to enable the virtual background.

    options

    Name
    Type
    Attributes
    Description
    sdkAppId
    number
    Required
    Current application ID
    userId
    string
    Required
    Current user ID
    userSig
    string
    Required
    UserSig corresponding to the user ID
    type
    string
    Optional
    image for image background
    blur for background blur (default)
    src
    string
    Required if type is image
    Image URL, such as https://picsum.photos/seed/picsum/200/300
    onError
    (error) => {}
    Optional
    Callback for errors that occur during runtime
    error.code=10000003 indicates high rendering latency
    error.code=10000006 indicates insufficient browser feature support, which may lead to lagging.
    Recommended solutions can be found in the Common Issues section at the end of the document.
    Example:
    await trtc.startPlugin('VirtualBackground', {
    sdkAppId: 123123,
    userId: 'userId_123',
    userSig: 'your_userSig',
    type: 'image',
    src: 'https://picsum.photos/seed/picsum/200/300'
    });

    trtc.updatePlugin('VirtualBackground', options)

    Allows modification of virtual background parameters.

    options

    Name
    Type
    Attributes
    Description
    type
    string
    Required
    image for image background
    blur for background blur
    src
    string
    Required if type is image
    Image URL, such as https://picsum.photos/seed/picsum/200/300
    Example:
    await trtc.updatePlugin('VirtualBackground', {
    type: 'blur'
    });

    trtc.stopPlugin('VirtualBackground')

    Disables the virtual background.

    Frequently Asked Questions

    When running the demo in Chrome, the video appears upside down and laggy?

    This plugin uses GPU acceleration, and you need to enable hardware acceleration mode in your browser settings. You can copy and paste chrome://settings/system into your browser's address bar and enable hardware acceleration mode.

    When device performance is insufficient and causes high latency with long rendering times?

    You can reduce video resolution or frame rate by listening to events.
    async function onError(error) {
    const { code } = error;
    if (code === 10000003 || code === 10000006) {
    // Reduce resolution and frame rate
    await trtc.updateLocalVideo({
    option: {
    profile: '480p_2'
    },
    });
    // await trtc.stopPlugin('VirtualBackground'); // Or disable the plugin
    }
    }
    
    await trtc.startPlugin('VirtualBackground', {
    ...// Other parameters
    onError,
    });

    How to start the plugin faster? (Manually deploy virtual background model files)

    The principle of this plugin is: after capturing from the camera, perform portrait segmentation locally through machine learning models, and then upload the results to the backend.
    When first enabled, it will automatically download machine learning models from Tencent Cloud CDN, but Tencent Cloud CDN and the static file address of your webpage are not from the same-origin, which will reduce some loading speeds. The best solution is to deploy model files on your own static resource server, such as deploying them together with web services.
    The acceleration method is very simple, unzip assets.zip into your project, such as ./src/assets/, and specify this address when creating a TRTC instance:
    const trtc = TRTC.create({ plugins: [VirtualBackground], assetsPath: './src/assets/' });
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support