- Release Notes and Announcements
- Release Notes
- Announcements
- Announcement on the Official Launch of the VOD Scene Transcoding Feature
- Announcement about the addition of mobile player SDK premium features authorization verification
- Announcement about the addition of web player SDK license authorization verification
- Announcement on the Official Launch of the Video on Demand Real-Time Log Analysis Feature
- Notice about the adjustment of the VOD adding domain feature
- Announcement on official commercial billing of Video on Demand STANDARD_IA data retrieval and Application Management functions
- Announcement on the Official Launch of the Media Quality Inspection and Play Channel Features
- Announcement on the official launch of the Video-on-Demand QUIC Acceleration feature
- VOD Launched Remaster Feature
- Prepaid Packages Supported for Monthly Billed Accounts
- Cost Allocation by Tag Supported
- Content Moderation Upgraded
- VOD Player Signature Upgrade
- Live Clipping to Become Paid Feature
- Content Recognition to Become Paid Feature
- The “startPlay” API of the Player SDK Renamed
- TESHD Renamed as TSC
- Announcement on VOD Video Acceleration Price Change
- Launching Video on Demand (VOD) Features for Regions Outside the Chinese Mainland
- About Changing the Billing Rule of Transcoding Service for VOD/MPS
- Product Introduction
- Purchase Guide
- Getting Started
- Console Guide
- Console Overview
- Service Overview
- Application Management
- Media Management
- Package Management
- License Management
- Real-Time Log Analysis
- Practical Tutorial
- Media Upload
- Smart Cold Storage of VOD Media Asset Files
- Video Processing
- Distribution and Playback
- How to Receive Event Notification
- How to Migrate Files from Origin Server to VOD
- Live Recording
- How to Make VOD Videos Live Streaming-Like
- Using VOD and WordPress to Build a Website
- How to Pull from Custom Origin Servers
- Development Guide
- Player SDK
- Server APIs
- History
- Introduction
- API Category
- Other APIs
- Media processing related API
- No longer recommended APIs
- Task Management APIs
- Media Upload APIs
- Media Management APIs
- Event Notification Relevant API
- Media Categorization APIs
- Distribution APIs
- AI-based Sample Management APIs
- Region Management APIs
- Statistics APIs
- Just In Time Transcode APIs
- Making API Requests
- AI-based image processing APIs
- Parameter Template APIs
- CreateWatermarkTemplate
- CreateSnapshotByTimeOffsetTemplate
- CreateSampleSnapshotTemplate
- CreateImageSpriteTemplate
- CreateAnimatedGraphicsTemplate
- CreateAdaptiveDynamicStreamingTemplate
- CreateAIAnalysisTemplate
- ModifyWatermarkTemplate
- ModifySnapshotByTimeOffsetTemplate
- ModifySampleSnapshotTemplate
- ModifyImageSpriteTemplate
- ModifyAnimatedGraphicsTemplate
- ModifyAdaptiveDynamicStreamingTemplate
- ModifyAIAnalysisTemplate
- DescribeTranscodeTemplates
- DescribeWatermarkTemplates
- ModifyAIRecognitionTemplate
- DescribeSnapshotByTimeOffsetTemplates
- DescribeSampleSnapshotTemplates
- DescribeImageSpriteTemplates
- DescribeAnimatedGraphicsTemplates
- DescribeAdaptiveDynamicStreamingTemplates
- DescribeAIAnalysisTemplates
- DeleteTranscodeTemplate
- DescribeAIRecognitionTemplates
- DeleteWatermarkTemplate
- DeleteSnapshotByTimeOffsetTemplate
- DeleteSampleSnapshotTemplate
- DeleteImageSpriteTemplate
- DeleteAnimatedGraphicsTemplate
- DeleteAdaptiveDynamicStreamingTemplate
- DeleteReviewTemplate
- DeleteAIRecognitionTemplate
- DeleteAIAnalysisTemplate
- DeleteHeadTailTemplate
- DeleteImageProcessingTemplate
- DescribeReviewTemplates
- CreateReviewTemplate
- CreateTranscodeTemplate
- ModifyReviewTemplate
- ModifyTranscodeTemplate
- CreateHeadTailTemplate
- CreateImageProcessingTemplate
- CreateAIRecognitionTemplate
- DescribeHeadTailTemplates
- DescribeImageProcessingTemplates
- ModifyHeadTailTemplate
- CreateQualityInspectTemplate
- DeleteQualityInspectTemplate
- DescribeQualityInspectTemplates
- ModifyQualityInspectTemplate
- CreateEnhanceMediaTemplate
- DeleteEnhanceMediaTemplate
- ModifyEnhanceMediaTemplate
- DescribeEnhanceMediaTemplates
- Task Flow APIs
- Domain Name Management APIs
- Playlist APIs
- Data Types
- Error Codes
- FAQs
- Agreements
- VOD Policy
- Contact Us
- Glossary
- Release Notes and Announcements
- Release Notes
- Announcements
- Announcement on the Official Launch of the VOD Scene Transcoding Feature
- Announcement about the addition of mobile player SDK premium features authorization verification
- Announcement about the addition of web player SDK license authorization verification
- Announcement on the Official Launch of the Video on Demand Real-Time Log Analysis Feature
- Notice about the adjustment of the VOD adding domain feature
- Announcement on official commercial billing of Video on Demand STANDARD_IA data retrieval and Application Management functions
- Announcement on the Official Launch of the Media Quality Inspection and Play Channel Features
- Announcement on the official launch of the Video-on-Demand QUIC Acceleration feature
- VOD Launched Remaster Feature
- Prepaid Packages Supported for Monthly Billed Accounts
- Cost Allocation by Tag Supported
- Content Moderation Upgraded
- VOD Player Signature Upgrade
- Live Clipping to Become Paid Feature
- Content Recognition to Become Paid Feature
- The “startPlay” API of the Player SDK Renamed
- TESHD Renamed as TSC
- Announcement on VOD Video Acceleration Price Change
- Launching Video on Demand (VOD) Features for Regions Outside the Chinese Mainland
- About Changing the Billing Rule of Transcoding Service for VOD/MPS
- Product Introduction
- Purchase Guide
- Getting Started
- Console Guide
- Console Overview
- Service Overview
- Application Management
- Media Management
- Package Management
- License Management
- Real-Time Log Analysis
- Practical Tutorial
- Media Upload
- Smart Cold Storage of VOD Media Asset Files
- Video Processing
- Distribution and Playback
- How to Receive Event Notification
- How to Migrate Files from Origin Server to VOD
- Live Recording
- How to Make VOD Videos Live Streaming-Like
- Using VOD and WordPress to Build a Website
- How to Pull from Custom Origin Servers
- Development Guide
- Player SDK
- Server APIs
- History
- Introduction
- API Category
- Other APIs
- Media processing related API
- No longer recommended APIs
- Task Management APIs
- Media Upload APIs
- Media Management APIs
- Event Notification Relevant API
- Media Categorization APIs
- Distribution APIs
- AI-based Sample Management APIs
- Region Management APIs
- Statistics APIs
- Just In Time Transcode APIs
- Making API Requests
- AI-based image processing APIs
- Parameter Template APIs
- CreateWatermarkTemplate
- CreateSnapshotByTimeOffsetTemplate
- CreateSampleSnapshotTemplate
- CreateImageSpriteTemplate
- CreateAnimatedGraphicsTemplate
- CreateAdaptiveDynamicStreamingTemplate
- CreateAIAnalysisTemplate
- ModifyWatermarkTemplate
- ModifySnapshotByTimeOffsetTemplate
- ModifySampleSnapshotTemplate
- ModifyImageSpriteTemplate
- ModifyAnimatedGraphicsTemplate
- ModifyAdaptiveDynamicStreamingTemplate
- ModifyAIAnalysisTemplate
- DescribeTranscodeTemplates
- DescribeWatermarkTemplates
- ModifyAIRecognitionTemplate
- DescribeSnapshotByTimeOffsetTemplates
- DescribeSampleSnapshotTemplates
- DescribeImageSpriteTemplates
- DescribeAnimatedGraphicsTemplates
- DescribeAdaptiveDynamicStreamingTemplates
- DescribeAIAnalysisTemplates
- DeleteTranscodeTemplate
- DescribeAIRecognitionTemplates
- DeleteWatermarkTemplate
- DeleteSnapshotByTimeOffsetTemplate
- DeleteSampleSnapshotTemplate
- DeleteImageSpriteTemplate
- DeleteAnimatedGraphicsTemplate
- DeleteAdaptiveDynamicStreamingTemplate
- DeleteReviewTemplate
- DeleteAIRecognitionTemplate
- DeleteAIAnalysisTemplate
- DeleteHeadTailTemplate
- DeleteImageProcessingTemplate
- DescribeReviewTemplates
- CreateReviewTemplate
- CreateTranscodeTemplate
- ModifyReviewTemplate
- ModifyTranscodeTemplate
- CreateHeadTailTemplate
- CreateImageProcessingTemplate
- CreateAIRecognitionTemplate
- DescribeHeadTailTemplates
- DescribeImageProcessingTemplates
- ModifyHeadTailTemplate
- CreateQualityInspectTemplate
- DeleteQualityInspectTemplate
- DescribeQualityInspectTemplates
- ModifyQualityInspectTemplate
- CreateEnhanceMediaTemplate
- DeleteEnhanceMediaTemplate
- ModifyEnhanceMediaTemplate
- DescribeEnhanceMediaTemplates
- Task Flow APIs
- Domain Name Management APIs
- Playlist APIs
- Data Types
- Error Codes
- FAQs
- Agreements
- VOD Policy
- Contact Us
- Glossary
Since August 1, 2022, the audio/video content recognition feature of VOD is paid feature. For more information, see Video Recognition to Become Paid Feature.
Audio/Video content recognition is an offline task that intelligently recognizes audio/video content with the aid of AI. It recognizes faces, text, opening and closing segments, and speech in the video, helping you accurately and efficiently manage your audio/videos. Specifically, it includes the following features:
Feature | Description | Use Cases |
Face recognition | Recognizes faces in video images. | Marks where celebrities appear in video images Checks for particular people in video images |
Full speech recognition | Recognizes all words that occur in speech | Generates subtitles for speech content Performs data analysis on video speech content |
Full text recognition | Recognizes all text that occurs in video images | Performs data analysis on text in video images |
Speech keyword recognition | Recognizes keywords in speech | Checks for sensitive words in speech Retrieves specific keywords in speech |
Text keyword recognition | Recognizes keywords in video images | Checks for sensitive words in video images Retrieves specific keywords in video images |
Opening and closing segment recognition | Recognizes opening and closing segments in videos | Marks the positions of the opening segment, and closing segment, and feature presentation in the progress bar Removes the opening and closing segments of multiple videos at a time |
Speech translation recognition | Recognizes all words that occur in speech and translates them into the specified language. | Generates translated subtitles for short dramas. Generates multilingual subtitles for recorded files of cross-border audio and video conferences. |
Some content recognition features depend on a material library. There are two types of libraries: public library and custom library.
Public library: VOD's preset material library.
Custom library: Your own library
Recognition Type | Public Library | Custom Library |
Face recognition | Supported. The library includes celebrities in the sports and entertainment industries, as well as other people. | |
Speech recognition | Not supported yet | |
Text recognition | Not supported yet |
Audio/Video Content Recognition Template
Audio/Video content recognition integrates a number of recognition features. You can use parameters to control the following:
Which content recognition features to enable
Whether to use the public library or custom library for face recognition
The confidence score threshold to return face recognition results
The labels of the faces to return
VOD provides preset video content recognition templates for common parameter combinations. You can also use a server API to create and manage custom templates.
Initiating a Task
You can initiate an audio/video recognition task by calling a server API, via the console, or by specifying the task when uploading videos. For details, see Task Initiation.
Below are the details:
Initiate a task by calling a server API: Call ProcessMedia, setting
Definition in the request parameter AiRecognitionTask to the ID of the audio/video content recognition template.Initiate a task via the console: Call the server API CreateProcedureTemplate to create an audio/video recognition task flow (
MediaProcessTask.AiRecognitionTask), and use it to process videos in the console.Specify a task when uploading videos from the server: Call the server API CreateProcedureTemplate to create an audio/video recognition task flow (
MediaProcessTask.AiRecognitionTask). When calling ApplyUpload, set the parameter procedure to the task flow.Specify a task when uploading videos from a client: Call the server API CreateProcedureTemplate to create an audio/video recognition task flow (
MediaProcessTask.AiRecognitionTask). When generating a signature for upload, set the parameter procedure to the task flow.Specify a task when uploading videos via the console: Call the server API CreateProcedureTemplate to create an audio/video recognition task flow (
MediaProcessTask.AiRecognitionTask). When uploading videos via the console, select Auto-processing after upload and choose the task flow.Getting the Result
After initiating an audio/video content recognition task, you can wait for the result notification asynchronously or perform a task query synchronously to get the task execution result. Below is an example of getting the result notification in normal callback mode after a content recognition task is initiated (the fields with null value are omitted):
{"EventType":"ProcedureStateChanged","ProcedureStateChangeEvent":{"TaskId":"1400155958-Procedure-2e1af2456351812be963e309cc133403t0","Status":"FINISH","FileId":"5285890784363430543","FileName":"Collection","FileUrl":"http://1400155958.vod2.myqcloud.com/xxx/xxx/aHjWUx5Xo1EA.mp4","MetaData":{"AudioDuration":243,"AudioStreamSet":[{"Bitrate":125599,"Codec":"aac","SamplingRate":48000}],"Bitrate":1459299,"Container":"mov,mp4,m4a,3gp,3g2,mj2","Duration":243,"Height":1080,"Rotate":0,"Size":44583593,"VideoDuration":243,"VideoStreamSet":[{"Bitrate":1333700,"Codec":"h264","Fps":29,"Height":1080,"Width":1920}],"Width":1920},"AiRecognitionResultSet":[{"Type":"FaceRecognition","FaceRecognitionTask":{"Status":"SUCCESS","ErrCode":0,"Message":"","Input":{"Definition":10},"Output":{"ResultSet":[{"Id":183213,"Type":"Default","Name":"John Smith","SegmentSet":[{"StartTimeOffset":10,"EndTimeOffset":12,"Confidence":97,"AreaCoordSet":[830,783,1030,599]},{"StartTimeOffset":12,"EndTimeOffset":14,"Confidence":97,"AreaCoordSet":[844,791,1040,614]}]},{"Id":236099,"Type":"Default","Name":"Jane Smith","SegmentSet":[{"StartTimeOffset":120,"EndTimeOffset":122,"Confidence":96,"AreaCoordSet":[579,903,812,730]}]}]}}}],"TasksPriority":0,"TasksNotifyMode":""}}
In the callback result,
ProcedureStateChangeEvent.AiRecognitionResultSet contains the result of face recognition (Type is FaceRecognition).According to the content of
Output.ResultSet, two people are recognized: John Smith and Jane Smith. SegmentSet indicates when (from StartTimeOffset to EndTimeOffset) and where (coordinates specified by AreaCoordSet) the two people appear in the video.
Yes
No
Was this page helpful?