tencent cloud

Feedback

Tencent Cloud AI Digital Human

HTML5 SDK API Description for Client Rendering

Last updated: 2024-11-07 09:51:58
Note:
Current version: 3.5.6

SDK Package File Description

The SDK file TXIVHSDK_Web_Client_Vx.x.x_Release.js.
A folder named unzip (available since version 3.4.0): It contains unzip_worker.js, unzip_wasm.js, and unzip_wasm_bg.wasm and is used for decompressing ZIP files. For details, see Initializing the Decompression Feature.

SDK API/Event Description

setPrivatization Settings

Set it before the init method. After settings, you do not need to pass in the sign parameter when calling the init method. If it is not set, you need to pass in the sign parameter.
Note: It needs to be set in a private deployment environment.

Parameter description

Parameter name
Required
Type
Description
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
appKey
Yes
String
See Product Introduction to obtain the key. After this parameter is set, you can initiate authentication locally.
accessToken
Yes
String
See Product Introduction to obtain the key. After this parameter is set, you can initiate authentication locally.
httpOrigin
Yes
String
Optional configuration with a high priority. If this parameter is set, all HTTP requests will be sent according to this origin. If it is not set, requests will be sent via the public cloud address.
wsOrigin
Yes
String
Optional configuration with a high priority. If this parameter is set, all socket requests will be sent according to this origin. If it is not set, requests will be sent via the public cloud address.

Return value description

Parameter name
Required
Type
Description
data
Yes
Boolean
true: The setting is successful.
false: The setting failed.

Sample code

const result = IVH.setPrivatization({
appKey: 'appkey in the key',
accessToken: 'accesstoken in the key',
httpOrigin: 'http://www.test.com',
wsOrigin: 'ws://www.test.com',
});
console.log(result);

Init (Initialization)

Note:
The ZIP file paths can be passed in since version 3.4.0, which can reduce the network file loading time. For details, see Initializing the Decompression Feature. (ZIP file paths can be passed in only after initUnzip is called.)

Parameter description

Parameter name
Required
Type
Description
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
element
Yes
Element/String
Container element object, or selector for finding the container element, for example, "#element ID".
virtualmanProjectId
Yes
String
For details, see Usage Process.
sign
Yes
String
For details, see Product Introduction.
If this parameter is not set, appKey and accessToken need to be set in setPrivatization.
If no setPrivatization is available and the sign parameter is set, public cloud authentication can be conducted.
modelPath
Yes
String
Set either this parameter or modelData. This parameter has a lower priority than other parameters.
Model path. Both relative and absolute paths are supported.
Note:
ZIP links are supported. For details, see Initializing the Decompression Feature.
modelData
Yes
ArrayBuffer
Set either this parameter or modelPath. This parameter has a higher priority than other parameters.
Model data. After fetching data, you can convert it to an ArrayBuffer.
actionPath
Yes
Array
Set either this parameter or actionData. This parameter has a lower priority than other parameters.
Action path. Both relative and absolute paths are supported.
Note:
ZIP links are supported. For details, see Initializing the Decompression Feature.
When the ZIP link of an action file is passed in, the first value in the array can be the URL, and other values will be ignored.

When the ZIP file is used:
Set silent action: Rename the corresponding_action.json file in the ZIP package to default_action_idle.json.
Set opening action: Rename the corresponding_action.json file in the ZIP package to opening_action_once.json.
actionData
Yes
Array
Set either this parameter or actionPath. This parameter has a higher priority than other parameters.
Action data.
defaultActionIdx
Yes
Int
The subscript in the action array, which will be regarded as the silent action. The default value is 0.
Note:
This parameter does not take effect when an action file in ZIP format is passed in.
openingActionIdx
Yes
Int
The subscript in the action array, which should not be the same as the silent action and should have a valid value. It will be executed once after initialization.
Note:
This parameter does not take effect when an action file in ZIP format is passed in.
modelOption
Yes
ModelOption
The model rendering parameter.
configPath
Yes
String
Set either this parameter or configData. This parameter has a lower priority than other parameters. Configuration file path. Both relative and absolute paths are supported.
Note:
If the downloaded model asset package contains a configuration file, you need to load it during initialization.
configData
Yes
Object
Set either this parameter or configPath. This parameter has a higher priority than other parameters. JSON data. After fetching data, you can convert it to a JSON file.
Note:
If the downloaded model asset package contains a configuration file, you need to load it during initialization.
isIntl
Yes
Boolean
Whether International Edition is used. The default value is false (Chinese Edition).
Warning:
Chinese | International Edition accounts and APIs are not mutually compatible. If they do not match, the API will be abnormal.



ModelOption Type Description

Parameter name
Required
Type
Description
mode
Yes
Array<String>
Mode collection.
Note:
position': Positioning mode. The position can be adjusted freely after configuration. Example: ['position'].

Sample code

// Path loading method.
IVH.init({
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
modelPath: `./model/yangxiaoyun/200.glb`,
actionPath: [
`./model/yangxiaoyun/action/hands_drooping.json`,
],
defaultActionIdx: 0
});

// Data loading method.
const option = {
modelPath: `./model/yangxiaoyun/200.glb`,
actionPath: [
`./model/yangxiaoyun/action/welcome.json`,
`./model/yangxiaoyun/action/thinking.json`,
],
};
const fetchArr = option.actionPath.map(n => {
return fetch(n).then(d => d.json());
});
Promise.all([fetch(option.modelPath).then(d => d.arrayBuffer())].concat(fetchArr)).then(data => {
const opt = {
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
defaultActionIdx: 0,
};
opt.modelData = data[0];
opt.actionData = [
{
name: 'welcome',
data: data[1],
},
{
name: 'thinking',
data: data[2],
}
];
IVH.init(opt);
});

Loading Action

Note:
This action should be executed after initialization.

Parameter description

Parameter name
Required
Type
Description
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
actionPath
Yes
Array
Set either this parameter or actionData. This parameter has a lower priority than other parameters.
Action path. Both relative and absolute paths are supported.
actionData
Yes
Array
Set either this parameter or actionPath. This parameter has a higher priority than other parameters.
Action data.

Sample code

IVH.on('init', e => {
console.log('init:', e);
IVH.loadAction({
actionPath: [
`./model/yangxiaoyun/action/welcome.json`,
],
});
});

Initializing AudioContext

Sample code

Note:
Due to iOS's audio auto-play restrictions, you should call the init method in the callback of the behavior event:
Correct example: Call initAudio at the level of the user behavior event callback function.
document.querySelector('#btn').onclick = function (e) {
IVH.initAudio();
const txt = document.querySelector('#txt').value.trim();
if (txt) {
setTimeout(() => {
IVH.play(txt);
}, 1000);
}
}
Incorrect example: Call initAudio at other function levels. In this case, iOS will consider it non-user behavior.
document.querySelector('#btn').onclick = function (e) {
const txt = document.querySelector('#txt').value.trim();
if (txt) {
setTimeout(() => {
IVH.initAudio();
IVH.play(txt);
}, 1000);
}
}

Play (Broadcast)

Note:
Execute initAudio before each call.

Parameter description

Parameter name
Required
Type
Description
text
Yes
Object
Text to be played, which supports the SSML specification. (Pure action tags only support single actions.)
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
isChat
Yes
Boolean
Whether to enable the dialogue mode. The default value is false (plain text-driven).
When this parameter is set to true, the corresponding dialogue mode is enabled. The dialogue modes include customer service dialogue and large model dialogue.
isNew
Yes
Boolean
This parameter is valid when isChat is true.
Whether the new and old sessions belong to the same session (multi-round). The settings will affect the results of multi-round sessions.
isStream
Yes
Boolean
This parameter is valid when isChat is false.
When this parameter is set to true, the streaming text-driven mode is used. The default value is false.
Warning:
Streaming text does not support the SSML format.
seqNo
Yes
Int
This parameter should be passed when isStream takes effect.
Serial number of a streaming text shard, which starts from 1.
isFinal
Yes
Boolean
This parameter should be passed when isStream takes effect.
End marker of streaming text, which should be passed in for each streaming text segment.
streamId
Yes
String
This parameter should be passed when isStream takes effect.
Request ID of streaming text, which is a 32-character UUID used to identify the same stream.
userId
Yes
String
This parameter is valid when isChat is true. If this parameter is passed in, isNew will be invalid. You can manually identify whether it belongs to the same session (used for multi-terminal queries).
Note:
The value of this parameter is a string of no more than 64 characters, which is passed in by customers during the large model dialogue.

Sample code

document.querySelector('#btn').onclick = function (e) {
const txt = document.querySelector('#txt').value.trim();
if (txt) {
IVH.initAudio();
IVH.play(txt, {
isChat: true,
});
}
}

// Simulated streaming API test case, which is for reference only.
document.querySelector('#btn7').addEventListener('click', e => {
const arr = [
'The weather is nice today.',
'It is sunny and breezy.',
'We can go out and have fun.'
];
IVH.initAudio();
const streamId = crypto.randomUUID().replace(/-/g, '')
for (let i = 0; i < arr.length; i++) {
IVH.play(arr[i], {
isStream: true,
seqNo: i + 1,
isFinal: i === arr.length - 1,
streamId,
})
}
})

Version Information

Returned values

Parameter name
Required
Type
Description
result
Yes
Object
A JSON object that also contains version information objects.
{
version: '1.0.0'
}

Sample code

/**
* @returns Object
* * @property {String} version: version information, for example, '1.0.0'.
*/
IVH.info()

Setting volume level

Parameter description

Parameter name
Required
Type
Description
val
Yes
Number
Volume, which ranges from 0 to 1.

Sample code

let volume = 1;
document.querySelector('#btn2').onclick = function (e) {
volume = 1 - volume;
IVH.setVolume(volume);
}

Interrupting the Broadcast

Sample code

document.querySelector('#btn3').onclick = function (e) {
IVH.stop();
}

Obtaining the Instruction Configuration

This method has been supported since version 3.1.0.

Parameter description

Parameter name
Required
Type
Description
key
Yes
String
Instruction key. You can obtain the configured instruction information based on it.

Returned values

Parameter name
Required
Type
Description
result
Yes
Promise<Object>
JSON object returned asynchronously. For details about key fields, see Appendix Description.

Sample code

const result = await IVH.getActionConfig('xxx');
console.log(result);

Obtaining the Configuration Information

This method has been supported since version 3.2.0.

Returned values

Parameter name
Required
Type
Description
result
Yes
Object
JSON data is returned. It is generally saved as conf.json after the model position is adjusted. During the next initialization, you can load it to adjust the initial model position.

Sample code

const result = IVH.getConf();
console.log(result);

Resetting the Position Information

This method has been supported since version 3.2.0. If you have customized a position during initialization, you can call this method to reset the position.

Sample code

IVH.resetPositionInfo();

Destroy

This method has been supported since version 3.3.0. It is used to terminate instances to free up memory.
Note:
After termination, if you want to render again, you need to re-initialize the corresponding method.

Sample code

IVH.destroy();

Initializing the Decompression Feature

This method has been supported since version 3.4.0. After initialization, the model and action files can be compressed into the ZIP package, reducing network request time.
Note:
Only ZIP files and the Deflated compression algorithm are supported.
iOS 11.0 or above and Android 7.0 or above are supported.

Parameter description

Parameter name
Required
Type
Description
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
path
Yes
String
The path where files for decompression are located.
There are three such files, namely unzip_worker.js, unzip_wasm.js, and unzip_wasm_bg.wasm. Place them in the same folder and pass the path, for example, http://xxx.com/unzip/, or a relative path, such as ./unzip/, to the path parameter.

Returned values

Parameter name
Required
Type
Description
result
Yes
Promise<Boolean>
Boolean value returned asynchronously. true indicates success, and false indicates failure. In case of a failure, the relevant error information will be returned in the error event.

Sample code

// Ensure that initUnzip returns true before the ZIP link is passed in.
(async () => {
const result = await IVH.initUnzip({
path: './unzip'
});
result && IVH.init({
element: '#xxxx',
sign: 'xxxxx',
virtualmanProjectId: 'xxxxx',
modelPath: './model/model.zip',
// For the ZIP link of an action file, only the URL in the array is considered, and subsequent values will be ignored. When using the ZIP file, you can specify a silent action by renaming the corresponding_action.json file in the ZIP package to default_action_idle.json. The system will consider this naming action as a silent action.
actionPath: [
'./model/action.zip',
],
});
})();

Loading Component Data

This method has been supported since version 3.5.0. After the init and initUnzip methods are called for a model, component data (for example, matching actions with the prop model) can be loaded.

Parameter description

Parameter name
Required
Type
Description
option
Yes
Option
Customized type

Option Type Description

Parameter name
Required
Type
Description
zipPath
Yes
String
The path where component compression files are located.
Due to the large number of files related to prop matching actions, if this parameter is not set, subsequent exceptions may occur. Therefore, file verification will be performed during loading.
In case of an exception, the relevant exception information will be returned in the error event. In case of a success, the relevant loading information will be returned in the actionLoad event.

Sample code

// Ensure that both init and initUnzip methods are called.
(async () => {
const result = await IVH.initUnzip({
path: './unzip'
});
IVH.on('init', e => {
IVH.loadComponent({
zipPath: './model/aiyun/aiyun.zip',
})
});
result && IVH.init({
element: '#xxxx',
sign: 'xxxxx',
virtualmanProjectId: 'xxxxx',
modelPath: './model/model.zip',
// When the ZIP link of an action file is passed in, the first value in the array can be the URL, and other values will be ignored
actionPath: [
'./model/action.zip',
],
});
})();

Listening for events

Parameter description

Parameter name
Required
Type
Description
event
Yes
String
Event
callback
Yes
Function
Callback function.

List of Listenable Events

Event
Required
Description
error
Yes
Triggered when an error occurs.
init
Yes
Triggered after model initialization is completed.
actionLoad
Yes
Triggered after action data is loaded.
actionEnd
Yes
Triggered after the action is executed.
play
Yes
Triggered after the TTS voice is played.
reskinEnd
Yes
Skin change event, which is triggered after skin change ends or when it is abnormal.
nlp
Yes
Dialogue return result, which is used for rich text display.
For details, see Appendix Description.
sentence
Yes
Sentence-based broadcast information in large model mode.
seqNo corresponds to seqNo returned in the NLP event. status indicates the playback status, including start and over.

Sample code

IVH.on('init', e => {
console.log('init:', e);
});


Guide

Switching Between Two Models to Achieve the Dressing Effect

Sample code

// Initialize clothing model A first.
IVH.init({
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
modelPath: `./model/modelA.glb`,
actionPath: [
`./model/actionA/xxxx.json`,
]
});

// Terminate the previous clothing model A before the switch.
IVH.destroy();
// Re-initialize clothing model B.
IVH.init({
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
modelPath: `./model/modelB.glb`,
actionPath: [
`./model/actionB/xxxx.json`,
]
});

Adjusting the Starting Position and Size of the Model

Sample code

// 1. Initialize the model and enable the positioning mode.
IVH.init({
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
modelPath: `./model/model.glb`,
actionPath: [
`./model/action/xxxx.json`,
],
modelOption: {
mode: ['position']
}
});

// 2. Scroll the mouse wheel to zoom in or out the model size, and right-click to drag the model position. After confirmation, call getConf to save the model as conf.json.
getConf.addEventListener('click', e => {
const blob = new Blob([JSON.stringify(IVH.getConf())], { type: 'application/json' });
// Name of the downloaded file.
const filename = 'conf.json';
// Create a hidden downloadable link.
const eleLink = document.createElement('a');
eleLink.download = filename;
eleLink.style.display = 'none';
// Convert the download content into a blob URL.
eleLink.href = URL.createObjectURL(blob);
// Trigger the click action.
document.body.appendChild(eleLink);
eleLink.click();
// Remove the URL.
document.body.removeChild(eleLink);
URL.revokeObjectURL(eleLink.href);
});

// 3. Re-initialize the model and load configPath/configData. The model will be rendered based on the previously saved information.
IVH.init({
element: '#model',
virtualmanProjectId: 'xxxxxxxxxxxxxxxxxx',
sign: 'xxxxxxxxxxxxxxxxxx',
modelPath: `./model/model.glb`,
actionPath: [
`./model/action/xxxx.json`,
],
configPath: './model/conf.json',
});

Appendix Description

Model

The following table describes the key fields returned by the NLP event.
Attribute Name
Description
replyType
cloudAiGpt: Tencent Cloud large model dialogue.
yunxiaowei: Tencent Cloud Xiaowei customer service.
cloudAiWaiting: Script for waiting for the first package due to timeout.
cloudAiTimeOut: Script for timeout without response and the session ends.
sensitive: Fixed script for sensitive content.
input: Content of the plain or streaming text input.
enhanceText: Plain text-driven content matches the content in script management.
cloudAiExtra
String in JSON text format, which is an additional supplementary field for large models.
This field has a value when replyType is cloudAiGpt and isFinal is true.
record_id
Message ID, which is the attribute of cloudAiExtra.
-- references
Answer source. For details.
seqNo
Serial number of a clause.
isFinal
Whether it is the last sentence.
isHighLight
Highlight tag.
contentType
Type in MD format.
0: unknown.
1: ordinary string.
2: ordered list.
3: unordered list.
4: image link.
5: HTTP link.
6: table.
7: code block.

Customer Service Dialogue

The following table describes the key fields returned by the NLP event. (These fields have been standardized to start with a lowercase letter since version 2.3.1.)
Attribute Name
Description
replyDisplay
Content configured at the configuration end, which uses the HTML syntax.
replyPro
TTS broadcast content, which uses the SSML syntax.
interactionType
Special message type. For details, see Description of Special Messages below.
interactionContent
Special message content. For details, see Description of Special Messages.

Description of Special Messages

Through the script management module of Tencent Cloud AI Digital Human (TCADH), you can configure special message types for different responses, assisting the integration party in quickly configuring special message content. The following are examples of codes for implementing front-end special message types. The SDK itself does not include front-end interaction styles.
Note:
The SDK itself does not include front-end interaction styles. If you need to use special message types, the integration party needs to develop the code.

Multiple Choice

/*
* Note: Style 1: style=1; style 2: style=2; style 3: style=3;
*/
{
InteractionContent: "{\\"style\\":1,\\"options\\":[\\"Option 1\\",\\"Option 2\\",\\"Option 3\\"]}",
InteractionType: "OptionInfo"
}

Image

{
InteractionContent: "{\\"url\\":\\"https://virtualhuman-cos-test-1251316161.cos.ap-nanjing.myqcloud.com/virtualman-config/0cbb2231-1afd-4cd5-8414-b1bc3d600a2f-Wang Bingbing.png\\"}",
InteractionType: "Image"
}

Pop-up window

{
InteractionContent: "{\\"title\\":\\"Pop-up headline\\",\\"content\\":\\"Pop-up content\\",\\"button\\":\\"Pop-up button\\"}",
InteractionType: "Popup"
}

Video

{
InteractionContent: "{\\"url\\":\\"https://virtualhuman-cos-test-1251316161.cos.ap-nanjing.myqcloud.com/virtualman-config/0417e732-1d64-4c65-a2c4-cac23ed454e2-1cd396fb-86a9-480f-814f-cbdf14055cdd.mp4\\"}",
InteractionType: "Video"
}

Image + Multiple Choice

/*
* Note: Style 1: style=1; style 2: style=2; style 3: style=3;
*/
{
InteractionContent: "{\\"url\\":\\"https://virtualhuman-cos-test-1251316161.cos.ap-nanjing.myqcloud.com/virtualman-config/22f4c069-cddb-4753-b482-02a20755b2ba-Wang Bingbing.png\\",\\"style\\":1,\\"options\\":[\\"Option 1\\",\\"Option 2\\"]}",
InteractionType: "ImageOption"
}

Custom

{
InteractionContent: "{\\"name\\":\\"Headline\\",\\"contents\\":[\\"Content\\"]}",
InteractionType: "Customize"
}



Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon