This document describes how to integrate with and debug GME client APIs for the voice messaging and speech-to-text services for Electron.
Key Considerations for Using GME
GME provides the real-time voice service and voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice service and voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, GmeError.AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Note:
There is a default call rate limit for speech-to-text APIs. For more information on how calls are billed within the limit, see Purchase Guide. If you want to increase the limit or learn more about how excessive calls are billed, submit a ticket. Non-streaming speech-to-text API PttSpeechToText(): There can be up to 10 concurrent requests per account.
Streaming speech-to-text API PttStartRecordingWithStreamingRecognition(): There can be up to 50 concurrent requests per account.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
TS class
`GmeContext`: GME business implementation APIs
`GmeError`: GME error code definition class
Core APIs
Importing the GME module
const { GmeContext } = require('gme-electron-sdk');
Getting an instance
var m_context = new GmeContext();
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
Init(appid: string, openid: string): number;
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized the SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code
The 7015 error code is judged by MD5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Sample code
number ret = m_context.Init(sdkAppId, openID);
// Determine whether the initialization is successful by the returned value
if (ret != GmeError.AV_OK)
{
console.log("Failed to initialize the SDK:");
return;
}
Triggering event callback
Event callbacks can be triggered by calling the Poll API in the timer. The Poll API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper file in SDK Download Guide. Note:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
Sample code
setInterval(function () {
m_context.Poll();
}, 50);
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Messaging and Speech-to-Text Services
Note:
The speech-to-text service consists of fast recording-to-text conversion and streaming speech-to-text conversion.
You do not need to enter a voice chat room when using the voice messaging service.
The maximum recording duration of a voice message is 58 seconds by default, and the minimum recording duration cannot be less than 1 second. If you want to customize the recording duration, for example, to modify the maximum recording duration to 10 seconds, call the SetMaxMessageLength API to set it after initialization.
Flowchart for using the voice message service
Flowchart for using the speech-to-text service
|
| Gets the authentication information |
| Specifies the maximum length of voice message |
Generating the local authentication key
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, use the backend deployment key as detailed in Authentication Key. API prototype
GenAuthBuffer(appId: string,roomId: string, openId:string, appKey: string) :string
|
| | AppId from the Tencent Cloud console.
|
| | Enter null or an empty string |
| | User ID, which is the same as OpenId during initialization. |
| | Permission key from the Tencent Cloud console. |
Application authentication
After the authentication information is generated, the authentication is assigned to the SDK.
API prototype
ApplyPTTAuthbuffer(authBuffer: string) :number
|
| | Authentication information |
Sample code
var authBuffer = m_context.GetAuthBuffer(UserConfig.GetAppID(), UserConfig.GetUserID(), null,UserConfig.GetAuthKey());
m_context.ApplyPTTAuthbuffer(authBuffer);
Specifying the maximum duration of voice message
This API is used to specify the maximum duration of a voice message, which can be up to 58 seconds.
API prototype
PttSetMaxMessageLength(msTime: number) :number
|
| | Audio duration in ms. Value range: 1000 < msTime <= 58000 |
Sample code
m_context.PttSetMaxMessageLength(58000);
Streaming Speech Recognition
Voice messaging and speech-to-text APIs
|
PttStartRecordingWithStreamingRecognition | Starts streaming recording |
| Stops recording |
Starting streaming speech recognition
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call Stop recording. API prototype
PttStartRecordingWithStreamingRecognition(filePath: string, speechLanguage: string, translateLanguage: string) :number
|
| | Path of stored audio file |
| | |
| | |
Sample code
string filePath = "xx/xxx/xxx.silk"
var ret = m_context.StartRecordingWithStreamingRecognition(filePath,"cmn-Hans-CN","cmn-Hans-CN");
if (ret == 0) {
this.currentStatus = "Start streaming recording";
} else {
this.currentStatus = "Failed to start streaming recording";
}
Note:
Translation incurs additional fees. For more information, see Purchase Guide. Callback for streaming speech recognition
After streaming speech recognition is started, you need to listen on callback messages in the OnEvent notification, which is as detailed below:
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING returns the recognized text in real time during the recording, which is equivalent to returning the recognized text while speaking.
The event message will be identified in the callback notification based on the actual needs. The passed parameters include the following four messages.
|
| Return code indicating whether streaming speech recognition is successful |
| Text converted from speech |
| Local path of stored recording file |
| Backend URL address of recording file, which will be retained for 90 days |
Note:
The file_id is empty when the 'ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRecognition_IS_RUNNING' message is listened.
Error codes
|
| Streaming speech-to-text conversion failed, but recording succeeded. | Call the UploadRecordedFile API to upload the recording file and then call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed, but recording and upload succeeded. | The message returned contains a backend URL after successful upload. Call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE:
{
HandleSTREAM2TEXTComplete(data,true);
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING:
{
HandleSTREAM2TEXTComplete(data, false);
break;
}
}
});
Voice Message Recording
The recording process is as follows: start recording > stop recording > return recording callback > start the next recording.
Voice messaging and speech-to-text APIs
Starting recording
This API is used to start recording.
API prototype
PttStartRecording(filePath: string) : number;
|
| | Path of stored audio file |
Sample code
string filepath = "xxxx/xxx.silk";
var ret = m_context.PttStartRecording(filepath);
Stopping recording
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
API prototype
PttStopRecording() :number;
Sample code
m_context.PttStopRecording();
Callback for recording start
A callback will be executed through a delegate function to pass a message when recording is completed.
To stop recording, call StopRecording. The callback for recording start will be returned after the recording is stopped.
|
| | 0: Recording is completed |
| | Path of stored recording file, which must be accessible and cannot be the fileid |
Error codes
|
| | Check whether the API parameters in the code are correct. |
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| Recording is in progress. | Ensure that the SDK recording feature is used at the right time. |
| Audio data is not captured. | Check whether the mic is working properly. |
| An error occurred while accessing the file during recording. | Ensure the existence of the file and the validity of the file path. |
| The mic is not authorized. | Mic permission is required for using the SDK. To add the permission, see the SDK project configuration document for the corresponding engine or platform. |
| The recording duration is too short. | The recording duration should be in ms and longer than 1,000 ms. |
| No recording operation is started. | Check whether the recording starting API has been called. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE:
{
// Process
break;
}
}
});
Pausing recording
This API is used to pause recording. If you want to resume recording, call the PttResumeRecording API.
API prototype
PttPauseRecording() : number
Sample code
number ret = m_context.PttPauseRecording();
Resuming recording
This API is used to resume recording.
API prototype
PttResumeRecording() : number;
Sample code
number ret = m_context.PttResumeRecording();
Canceling recording
This API is used to cancel recording. There is no callback after cancellation.
API prototype
PttCancelRecording() : number
Sample code
m_context.PttCancelRecording();
Voice Message Upload, Download, and Playback
|
| |
| |
| |
| Stops playing back an audio file |
| |
| Gets the audio file duration |
Uploading an audio file
This API is used to upload an audio file.
API prototype
PttUploadRecordedFile(filePath: string) : number
|
| | Path of uploaded audio file, which is a local path. |
Sample code
var ret = m_context.PttUploadRecordedFile(filePath);
Callback for audio file upload completion
After the audio file is uploaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.
The passed parameters include result, file_path, and file_id.
|
| | 0: Recording is completed
|
| | Path of stored recording file |
| | |
Error codes
|
| An error occurred while accessing the file during upload. | Ensure the existence of the file and the validity of the file path. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| A network error occurred. | Check whether the device can access the internet. |
| The network failed while getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| The packet returned during the process of getting the upload parameters is empty. | Check whether the authentication is correct and whether the device network can normally access the internet. |
| Failed to decode the packet returned during the process of getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| | Check whether the apply API is called or whether the input parameters are empty. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE:
{
// Process
break;
}
}
});
Downloading the audio file
This API is used to download an audio file.
API prototype
PttDownloadRecordedFile(fileId: string, filePath: string) : number
|
| | |
| | Local path of saved file, which must be accessible and cannot be the fileid |
Sample code
var ret = m_context.PttDownloadRecordedFile(fileID,filePath);
Callback for audio file download completion
After the audio file is downloaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.
The passed parameters include result, file_path, and file_id.
|
| | |
| | Path of stored recording file |
| | URL of recording file, which will be retained on the server for 90 days. |
Error codes
|
| An error occurred while accessing the file during download. | Check whether the file path is valid. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| Network storage system exception. | The server failed to get the audio file. Check whether the API parameter fileid is correct, whether the network is normal, and whether the file exists in COS. |
| Server file system error. | Check whether the device can access the internet and whether the file exists on the server. |
| The HTTP network failed during the process of getting the download parameters. | Check whether the device can access the internet. |
| The packet returned during the process of getting the download parameters is empty. | Check whether the device can access the internet. |
| Failed to decode the packet returned during the process of getting the download parameters. | Check whether the device can access the internet. |
| | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
}
});
Playing back audio
This API is used to play back audio.
API prototype
PttPlayRecordedFile(filePath: string, voiceType: ITMG_VOICE_TYPE) : number
Error codes
|
| | Ensure the existence of the file and the validity of the file path. |
Sample code
m_context.PlayRecordedFile(filePath);
Callback for audio playback
After the audio is played back, the event message ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE will be returned, which will be identified in the OnEvent function.
The passed parameter includes result and file_path.
|
| | |
| | Path of stored recording file |
Error codes
|
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| During playback, the client tried to interrupt and play back the next one but failed (which should succeed normally). | Check whether the code logic is correct. |
| | Check whether the API parameters in the code are correct. |
| | An error occurred while initializing the player. This error code is generally caused by failure in decoding, and the error should be located with the aid of logs. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE:
{
// Process
break;
}
}
});
Stopping audio playback
This API is used to stop audio playback. There will be a callback for playback completion when the playback stops.
API prototype
PttStopPlayFile() : number
Sample code
m_context.PttStopPlayFile();
Getting audio file size
This API is used to get the size of an audio file.
API prototype
PttGetFileSize(filePath: string) : number
|
| | Path of audio file, which is a local path |
Sample code
m_context.PttGetFileSize(filePath);
Getting audio file duration
This API is used to get the duration of an audio file in milliseconds.
API prototype
PttGetVoiceFileDuration(filePath: string) : number
|
| | Path of audio file, which is a local path |
Sample code
number fileDuration = m_context.PttGetVoiceFileDuration(filePath);
Fast Recording-to-Text Conversion
Translating audio file into text in specified language
This API can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation.
Note:
Translation incurs additional fees. For more information, see Purchase Guide. API prototype
PttSpeechToText(fileID: string, speechLanguage: string, translateLanguage: string) : number
|
| | URL of audio file, which will be retained on the server for 90 days |
| | |
| | |
Sample code
m_context.PttSpeechToText(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Callback for recognition
After the specified audio file is converted to text, the event message ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE will be returned, which will be identified in the OnEvent function.
The passed parameters include result, file_path and text (recognized text).
|
| | 0: Recording is completed
|
| | URL of recording file, which will be retained on the server for 90 days |
| | |
Error codes
|
| An internal error occurred. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the device can access the internet. |
| Failed to decode the returned packet. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
| | Check whether authbuffer is correct. |
| Incorrect speech-to-text conversion parameter. | Check whether the API parameter fileid in the code is empty. |
| Speech-to-text translation returned an error. | Error with the backend of voice messaging and speech-to-text feature. Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE:
{
// Process
break;
}
});
Voice Message Volume Level APIs
|
| Gets real-time mic volume level |
| Sets recording volume level |
| Gets recording volume level |
| Gets real-time speaker volume level |
| |
| Gets playback volume level |
Getting the real-time mic volume of voice message
This API is used to get the real-time mic volume. A number-type value will be returned. Value range: 0–200.
API prototype
Sample code
m_context.PttGetMicLevel();
Setting the recording volume of voice message
This API is used to set the recording volume of voice message. Value range: 0-200.
API prototype
PttSetMicVolume(vol:number) :number
|
| | Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
m_context.PttSetMicVolume(vol);
Getting the recording volume of voice message
This API is used to get the recording volume of voice message. A number-type value will be returned. Value range: 0–200.
API prototype
PttGetMicVolume() : number
Sample code
m_context.PttGetMicVolume();
Getting the real-time speaker volume of voice message
This API is used to get the real-time speaker volume. A number-type value will be returned. Value range: 0-200.
API prototype
PttGetSpeakerLevel() : number;
Sample code
m_context.PttGetSpeakerLevel();
Setting the playback volume of voice message
This API is used to set the playback volume of voice message. Value range: 0-200.
API prototype
PttSetSpeakerVolume(vol: number) : number
Sample code
m_context.PttSetSpeakerVolume(100);
Getting the playback volume of voice message
This API is used to get the playback volume of voice message. A number-type value will be returned. Value range: 0-200.
API prototype
PttGetSpeakerVolume() : number
Sample code
m_context.PttGetSpeakerVolume();
Advanced APIs
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
Sample code
string sdkVersion = m_context.GetSDKVersion();
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
SetLogLevel(level: number) : number
Parameter description
|
| | Sets the log level. TMG_LOG_LEVEL_NONE indicates not to log. Default value: TMG_LOG_LEVEL_INFO. |
level description:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
m_context.SetLogLevel(TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\GMEGLOBAL\GME\ProcessName |
API prototype
SetLogPath(logPath: string)
Sample code
string logDir = ""// Set a path by yourself
m_context.SetLogPath(logDir);
