- Product Overview
- Server API Integration
- Avatar aPaas API Calling Methods
- Avatar Image Customization and Voice Clone API Documentation - v 0.2.1
- Video Generation Service API Documentation - v 4.9.9
- Overview
- Digital Human aPaaS API Calling Methods
- Audio Production API
- Video Production API - Basic Edition
- Audio and Video Production Progress Query API
- Video Production API - Advanced Version
- Customer Resource Query Anchor API
- Querying All Images of a Specific Anchor
- Querying the Supported Timbres for VirtualmanKey (to Be Deprecated)
- Querying the Supported Actions for VirtualmanKey
- Appendix
- API Integration FAQs
- Interactive Digital Human Service API Documentation - v 5.0.3
- Overview
- Digital Human aPaaS API Calling Methods
- Live Session Management
- Creating a New Live Stream Session
- Using the Personal Asset Image to Create a Stream
- Querying the Session Status
- Starting Session
- Closing Live Session
- Querying the Conversation List under the Digital Human Project
- Querying the Conversion List under the Personal Asset Image
- Querying the Session List under UIN
- Instruction-driven
- Other APIs
- Error Code Description
- Personal Asset Management API Documentation - v 1.1.2
- Overview
- Digital Human aPaaS API Calling Methods
- Querying for Avatar List by Pagination API
- Querying Supported Timbres for Avatars (to be Deprecated)
- Querying Customer Service Asset Information
- Querying Timbre Lists by Pagination
- Querying Image Asset Information - Query Anchor
- Querying Image Asset Information - Querying all Avatars under the Anchor
- Querying the List Of Actions Supported by the Avatar
- Appendix 1 - Service Asset Type
- Appendix 2 - Emotional Style
- Appendix 3 - Digital Human Type
- API Integration FAQs
- FAQs
- Product Overview
- Server API Integration
- Avatar aPaas API Calling Methods
- Avatar Image Customization and Voice Clone API Documentation - v 0.2.1
- Video Generation Service API Documentation - v 4.9.9
- Overview
- Digital Human aPaaS API Calling Methods
- Audio Production API
- Video Production API - Basic Edition
- Audio and Video Production Progress Query API
- Video Production API - Advanced Version
- Customer Resource Query Anchor API
- Querying All Images of a Specific Anchor
- Querying the Supported Timbres for VirtualmanKey (to Be Deprecated)
- Querying the Supported Actions for VirtualmanKey
- Appendix
- API Integration FAQs
- Interactive Digital Human Service API Documentation - v 5.0.3
- Overview
- Digital Human aPaaS API Calling Methods
- Live Session Management
- Creating a New Live Stream Session
- Using the Personal Asset Image to Create a Stream
- Querying the Session Status
- Starting Session
- Closing Live Session
- Querying the Conversation List under the Digital Human Project
- Querying the Conversion List under the Personal Asset Image
- Querying the Session List under UIN
- Instruction-driven
- Other APIs
- Error Code Description
- Personal Asset Management API Documentation - v 1.1.2
- Overview
- Digital Human aPaaS API Calling Methods
- Querying for Avatar List by Pagination API
- Querying Supported Timbres for Avatars (to be Deprecated)
- Querying Customer Service Asset Information
- Querying Timbre Lists by Pagination
- Querying Image Asset Information - Query Anchor
- Querying Image Asset Information - Querying all Avatars under the Anchor
- Querying the List Of Actions Supported by the Avatar
- Appendix 1 - Service Asset Type
- Appendix 2 - Emotional Style
- Appendix 3 - Digital Human Type
- API Integration FAQs
- FAQs
Note:
Based on a persistent connection, instructions are sent to drive the AI digital human.
Persistent connection instructions are divided into sending text instructions, sending audio instructions, and sending heartbeat instructions.
Sending a text instruction involves sending broadcast text to the cloud, which synthesizes speech to drive the AI digital human to speak;
Sending an audio instruction involves sending voice stream shards to the cloud, which then uses the original sound or voice changer to drive the AI digital human to speak;
Sending a heartbeat instruction is to maintain a persistent connection and prevent session disconnection. If the client does not send a heartbeat and there is no valid data exchanged, the cloud will automatically disconnect the persistent connection after 3 minutes and close the session stream after 10 minutes.
The relationship between the DriverType parameter and the instruction types when creating a new stream is as follows:
DriverType 1: Text-driven; supporting sending text instructions (SEND_TEXT) and heartbeat instructions (SEND_HEARTBEAT).
DriverType 3: Audio-driven; supporting sending text instructions (SEND_TEXT), audio instructions (SEND_AUDIO), and heartbeat instructions (SEND_HEARTBEAT).
1. When a text is being broadcast, if you want to send audio, you need to send an interrupt with an empty text instruction until you receive a TextOver event, after which you can send the audio.
2. When the text is being broadcast and you want to broadcast new text, directly send the new text instruction. The current text broadcast will be interrupted, and upon completion, you will receive a TextOver event. The broadcast of the new text will start with a TextStart event and end with a TextOver event.
3. When an audio-driven session is in progress, if you want to interrupt the audio to send a text instruction, you need to send an audio instruction with IsFinal set to true to end the audio session. Only after receiving an AudioOver event can you send text or audio.
Yes
No
Was this page helpful?