tencent cloud

$0 14-Day TrialExperience EdgeOne for acceleration and security protection!

Feedback

Tencent Effect SDK
Last updated: 2023-05-18 16:31:37

Overview

This capability processes the OpenGL textures of the camera and outputs blendshape data that meets the standards of Apple ARKit. For details, see ARFaceAnchor. You can pass the data to Unity to drive your model or use the data to implement other features.

Integration (iOS)

Follow the directions in Integrating Tencent Effect SDK to integrate the Tencent Effect SDK.

API Calls

1. Enable the feature.
[self.beautyKit setFeatureEnableDisable:ANIMOJI_52_EXPRESSION enable:YES];
2. Configure the callback of facial keypoints.
Version 2.6.0 and earlier versions use the following method.
//XMagic.h
- (void)registerSDKEventListener:(id<YTSDKEventListener> _Nullable)listener;

@implementation listener
- (void)onYTDataEvent:(id)event
{
NSLog(@"YTData %@", event);
}
Version 3.0.0 uses the following method.
//XMagic.h
- (void)registerSDKEventListener:(id<YTSDKEventListener> _Nullable)listener;

- (void)onAIEvent:(id)event
{
NSDictionary *eventDict = (NSDictionary *)event;
if (eventDict[@"ai_info"] != nil) {
NSLog(@"ai_info %@",eventDict[@"ai_info"]);
}
}
onYTDataUpdate returns a JSON string structure that contains the information of up to 5 faces:
onAIEvent returns a JSON string structure that contains the information of up to 5 faces:
{
"face_info":[{
"trace_id":5,
"face_256_point":[
180.0,
112.2,
...
],
"face_256_visible":[
0.85,
...
],
"out_of_screen":true,
"left_eye_high_vis_ratio:1.0,
"right_eye_high_vis_ratio":1.0,
"left_eyebrow_high_vis_ratio":1.0,
"right_eyebrow_high_vis_ratio":1.0,
"mouth_high_vis_ratio":1.0,
"expression_weights":[
0.12,
-0.32
...
]
},
...
]
}

Field descriptions

trace_id: The face ID. If the faces obtained from a continuous video stream have the same face ID, they belong to the same person.
expression_weights: The real-time blendshape data. The array contains 52 elements. The value range of each element is 0-1.0. {&quot;eyeBlinkLeft&quot;,&quot;eyeLookDownLeft&quot;,&quot;eyeLookInLeft&quot;,&quot;eyeLookOutLeft&quot;,&quot;eyeLookUpLeft&quot;,&quot;eyeSquintLeft&quot;,&quot;eyeWideLeft&quot;,&quot;eyeBlinkRight&quot;,&quot;eyeLookDownRight&quot;,&quot;eyeLookInRight&quot;,&quot;eyeLookOutRight&quot;,&quot;eyeLookUpRight&quot;,&quot;eyeSquintRight&quot;,&quot;eyeWideRight&quot;,&quot;jawForward&quot;,&quot;jawLeft&quot;,&quot;jawRight&quot;,&quot;jawOpen&quot;,&quot;mouthClose&quot;,&quot;mouthFunnel&quot;,&quot;mouthPucker&quot;,&quot;mouthRight&quot;,&quot;mouthLeft&quot;,&quot;mouthSmileLeft&quot;,&quot;mouthSmileRight&quot;,&quot;mouthFrownRight&quot;,&quot;mouthFrownLeft&quot;,&quot;mouthDimpleLeft&quot;,&quot;mouthDimpleRight&quot;,&quot;mouthStretchLeft&quot;,&quot;mouthStretchRight&quot;,&quot;mouthRollLower&quot;,&quot;mouthRollUpper&quot;,&quot;mouthShrugLower&quot;,&quot;mouthShrugUpper&quot;,&quot;mouthPressLeft&quot;,&quot;mouthPressRight&quot;,&quot;mouthLowerDownLeft&quot;,&quot;mouthLowerDownRight&quot;,&quot;mouthUpperUpLeft&quot;,&quot;mouthUpperUpRight&quot;,&quot;browDownLeft&quot;,&quot;browDownRight&quot;,&quot;browInnerUp&quot;,&quot;browOuterUpLeft&quot;,&quot;browOuterUpRight&quot;,&quot;cheekPuff&quot;,&quot;cheekSquintLeft&quot;,&quot;cheekSquintRight&quot;,&quot;noseSneerLeft&quot;,&quot;noseSneerRight&quot;,&quot;tongueOut&quot;}
The other fields are facial keypoint information. Whether they are returned depends on the type of license you use. If you only need facial expression data, you can ignore those fields.
Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon