Log in to WeData console, click the left menuData Source Management. On the Data Source Management page, you can view and manage the data source information under this project.
The project includes two types of data sources: system sources and custom sources. System sources are data sources automatically generated after binding computational storage engines; custom sources are created by the user.
System Source
System sources are data sources automatically created when users bind compute engines in the project. Currently, system sources only support viewing data source connection information and editing; they do not support deletion, authorization, or recycle operations.
Binding EMR engine will automatically create EMR Hive Data Source, EMR HBase Data Source, EMR Kyuubi Data Source, EMR Impala Data Source, EMR Trino Data Source;
Binding DLC engine will automatically create DLC Data Source;
Binding TCHouse-P engine will automatically create TCHouse-P Data Source.
View connection information: Select the row of the data source and click View Connection Information.
Editing: Select the row where the data source is located and click Edit. Only the primary account can edit the display name.
Self-Definition Source
Self-Definition Source refers to the data sources imported by users in WeData. It currently supports more than 20 types of data sources, including MySQL, Kafka, HDFS, etc. Self-Definition Source supports viewing connection information, editing, deleting, authorizing, and recycling operations.
View connection information: Select the row of the data source and click View Connection Information.
Edit: Select the row of the data source and click Edit.
Except for Connection Type, Data Source Name, and Project, all other configuration items can be edited.
Cloud Instance: Data sources introduced through cloud instances can modify data source description, username, and password.
Connection String: Data sources introduced through connection strings can modify description, JDBC URL parameters, username, and password.
After editing the data source, click Start Test to test the connectivity of the edited data source. Then click OK to save the edits.
Delete
Select the data source and click Delete in the operation column.
Confirm the data source information and click Delete.
Authorization/Bulk Authorization
Select the data source and click Authorize in the operation column.
Select Authorized Objects, either users or projects.
Select the target user or project and click OK.
Recycle
Select the data source and click Recycle in the operation column.
Select the target user or project that needs to have permissions recycled, click Recycle and then Confirm in the popup.
Batch Transfer
Select the data source and click Batch Transfer in the top-left corner.
Select the target user to whom the permissions need to be transferred, click Confirm. After the transfer, the person-in-charge of the data source will be the transferee.
Create Relational Database Data Source
The current version of Tencent Cloud WeData supports creating over 10 types of relational databases. Tencent Cloud relational database products (e.g., TDSQL-C MySQL, TDSQL MySQL, TDSQL PostgreSQL, Tencent Cloud MySQL, etc.) support connection via cloud instances. General relational database data sources (e.g., MySQL, PostgreSQL, Oracle, SQL Server, etc.) support connection via connection strings.
Select Data Source Type
Click New Data Source and select the data source type, then click Next.
Configure Data Source: TDSQL-C MySQL
Taking TDSQL-C MySQL as an example to explain Tencent Cloud relational database products. Other data sources follow the same logic. TDSQL-C MySQL data source supports both cloud instance and connection string connection methods.
Create a data source via cloud instance.
Parameter
Description
Connection Type
Select the data source connection form: cloud instance or connection string
Associated Project
Affiliated project when the current data source was created
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Display name of the data source when used in the product. If not filled, the data source name will be displayed by default
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Getting instance
Select the region, instance name, and ID information of the TencentDB instance under the account
Database Name
Database name to be connected
Username
Username to connect to the database
Password
Password to connect to the database
Data Connectivity
Test whether the configured database can be connected
Note:
If the connectivity test fails, the data source can still be saved, but the data source will not be accessible.
Create a data source via connection string.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Display name of the data source when used in the product. If not filled, the data source name will be displayed by default
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
Supports two deployment modes: Self-Built Instance and Public Network Instance.Self-Built Instance is deployed on Tencent CVM, while Public Network Instanceis is in the client’s local IDC or other cloud resources, accessible via public network.
Region and Network
When selecting a self-built instance, you need to choose the region and vpcID where the data source instance resides
JDBC URL
Information for connecting to TDSQL-C MySQL data source instance, including host IP, port, database name, etc
Database Name
Database name to be connected
Username
Username to connect to the database
Password
Password to connect to the database
Data Connectivity
Test whether the configured database can be connected.
Note:
If the connectivity test fails, the data source can still be saved, but the data source will not be accessible.
Configure Data Source: PostgreSQL
General-purpose relational database data source, explained with PostgreSQL as an example. Other data sources follow the same principle. PostgreSQL data sources support only connection string method.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
Supports two deployment modes: self-built instances and public network instances. Self-built instances are deployed on Tencent CVM, while public network instances are in the client’s local IDC or other cloud resources, accessible via public network.
Region and Network
When selecting a self-built instance, you need to choose the data source's region and vpcID
JDBC URL
Connection string information for connecting to PostgreSQL data source instances, including host IP, port, database name, etc
Username
Username to connect to the database
Password
Password to connect to the database
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: IBM DB2
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Region and Network
Select the region, instance name, and ID information of the TencentDB instance under the account.
DB2 Model
Select the corresponding DB2 model. The current version supports DB2 iSeries/AS 400, DB2 for LUW, and DB2 z/OS models
IP and Port Number
Enter the IP and port number information for the DB2 data source
Username
Username to connect to the database
Password
Password to connect to the database
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Sybase
Sybase data sources only support connection strings as the connection method.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
Choose between self-built instances or public network instances
Region and Network
Select the region, instance name, and ID information of the TencentDB instance under the account.
IP and Port Number
Enter the IP and port number information of the Sybase data source respectively
Database Name
Database name to be connected
Username
Username to connect to the database
Password
Password to connect to the database
Data Connectivity
Test whether the configured database can be connected
Create a big data data source
The current version of Tencent Cloud WeData supports the creation of over 10 types of big data sources, including TCHouse-C and TCHouse-D, which are Tencent Cloud big data products.
Select Data Source Type
Click New Data Source and select the data source type, then click Next.
Configuring a Data Source: HDFS
WeData currently supports two methods to introduce HDFS data sources: through Tencent Cloud EMR cloud instances and connection strings.
Introduce data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
Only supports Tencent Cloud EMR
Getting instance
Select the region, instance name, and ID information of the EMR instance under the account
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Introduce data sources through connection strings
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Deployment Method
Currently, only HDFS in Tencent Cloud EMR is supported for addition
Region and Network
Select the region first, then select the EMR instance under that region. Click the refresh button on the right side to retrieve instances under that region again
defaultFS
Used to specify the default file system, and the default is the local file system
Extended Configuration
Optional: Extended configurations for HDFS connection string method
core-site.xml file
HDFS core-site.xml file
hdfs-site.xml file
HDFS hdfs-site.xml file
Authentication Method
Kerberos authentication for HDFS connection
keytab file
keytab file for Kerberos authentication
conf file
conf file for Kerberos authentication
principal
principal required for Kerberos authentication
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configuring a Data Source: HBase
WeData currently supports two methods to introduce HBase data sources: through Tencent Cloud EMR cloud instances and connection strings.
Introduce data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
The current version only supports Tencent Cloud EMR
Getting instance
Select the region, instance name, and ID information of the EMR instance under the account
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Introduce data sources through connection strings
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data source deployment mode
Currently, only HBase in Tencent Cloud EMR is supported for addition
Getting instance
Select the region first, then select the EMR instance under that region. Click the refresh button on the right side to retrieve instances under that region again
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configuring a Data Source: Hive
WeData currently supports introducing Hive data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
JDBC URL
Connection string information for connecting to Hive data sources
Username
Username to connect to the data source
Password
Password to connect to the data source
Metadata Collection
When selected, a metadata collection task will be automatically created in the data discovery module for the current data source
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Kudu
WeData currently supports introducing Kudu data sources through Tencent Cloud EMR and connection strings.
Introduce data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
The current version only supports Tencent Cloud EMR
Getting instance
Select the region, instance name, and ID information of the EMR instance under the account.
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Introduce data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Kudu server list
Address information for connecting to the Kudu server
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Doris
WeData currently supports introducing Doris DB and StarRocks data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
You can choose a self-built instance or a public network instance. For self-built instances, you need to enter the region and network. Public network instances do not require region and network information
Region and Network
Select the region, instance name, and ID information of the TencentDB instance under the account.
JDBC URL
Connection string information for connecting to Hive data sources
FE URL
Enter the FE HTTP address in the format: IP address : HTTP port. Separate multiple addresses with commas (,), for example: 172.17.16.3:8030,172.17.16.4:8030
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Iceberg
WeData currently supports introducing Iceberg data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Region and Network
Select the region, instance name, and ID information of the TencentDB instance under the account.
catalog
You can choose HIVE or HDFS
Metastore URL
Please enter the Metastore URL
JDBC URL
Connection string information for connecting to Hive data sources
Username
Username to connect to the data source
Password
Password to connect to the data source
Warehouse path
Please enter the warehouse path
core-site.xml
Supports uploading core-site.xml
hdfs-site.xml
Supports uploading hdfs-site.xml
hive-site.xml
Supports uploading hive-site.xml
Authentication Method
Kerberos authentication method is optional
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: DLC
WeData currently supports accessing DLC data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
JDBC URL
Connection string information for connecting to DLC data sources
Username
Username to connect to the data source
SecretId
Username to connect to the data source
SecretKey
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Clickhouse, Kyuubi, TChouse-X
WeData currently supports introducing Clickhouse, Kyuubi, and TCHouse-X data sources through cloud instances and connection strings.
Introduce data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
The current version only supports Tencent Cloud EMR
Getting instance
Select the region, instance name, and ID information of the TencentDB instance under the account
Database name
Target database name
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Introducing data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
JDBC URL
Connection string information for connecting to Hive data sources
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring a Data Source: Trino
Tencent Cloud WeData currently supports introducing Trino data sources via connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters.
Display name
Optional display name. If not filled, the data source name will be displayed.
Description(Optional)
Optional, description of this data source.
Data Source Permission
Project sharing means all members of the current data source project can use it; Individual Only and Administrator indicates that the data source is only available to the creator and project administrators.
Deployment Method
Supports deployment in both self-built instances and public network instances
Region and Network
Select the region, instance name, and ID information of the TencentDB instance under the account.
IP and Port Number
Optional, enter the IP address and port number of the current data source
catalog
Selectable catalog
Schema
Please enter Sybase
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring Data Source: Greenplum, GauseDB, Impala, Tbase, Gbase
For data source types: Greenplum, GauseDB, Impala, Tbase, Gbase, Tencent Cloud WeData currently supports connection string access.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
JDBC URL
Connection string information for connecting to Hive data sources
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Create a semi-structured data source
In the current version, Tencent Cloud WeData supports four types of semi-structured data sources: FTP, SFTP, COS, and REST API, with COS being a Tencent Cloud product.
Select Data Source Type
Click New Data Source and select the data source type, then click Next.
Configuring Data Source: FTP
Tencent Cloud WeData currently supports introducing FTP data sources via connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
You can choose a self-built instance or a public network instance. For self-built instances, you need to enter the region and network. Public network instances do not require region and network information
Region and Network
Select the region and the corresponding vpcid in the current region
IP and Port Number
Enter the IP and port number of the FTP data source
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring Data Source: SFTP
Tencent Cloud WeData currently supports introducing SFTP data sources via connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
You can choose a self-built instance or a public network instance. For self-built instances, you need to enter the region and network. Public network instances do not require region and network information
Region and Network
Select the region and the corresponding vpcid in the current region
IP and Port Number
Enter the IP and port number of the SFTP data source
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Configuring Data Source: COS
Tencent Cloud WeData currently supports introducing COS data sources via cloud instance method.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
You can choose a self-built instance or a public network instance. For self-built instances, you need to enter the region and network. Public network instances do not require region and network information
Region and Network
Select the region and the corresponding vpcid in the current region
secertId
Username to connect to the data source
secertKey
Password to connect to the data source
Bucket
Optional, add the access path of the current data source
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configuring Data Source: REST API
Tencent Cloud WeData currently supports introducing REST API data sources via connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
You can choose a self-built instance or a public network instance. For self-built instances, you need to enter the region and network. Public network instances do not require region and network information
Region and Network
Select the region and the corresponding vpcid in the current region
URL
Connection string information for connecting to data sources
Default Request Headers
Optional Request Headers
Authentication Method
Authentication methods can be Basic or Token
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Create NoSQL Data Source
The NoSQL data sources supported by the current version of Tencent Cloud WeData include Redis, Tencent Cloud ElasticSearch, ElasticSearch, Tencent Cloud MongoDB, MongoDB, and Graph Database. Among them, Tencent Cloud ElasticSearch and Tencent Cloud MongoDB are Tencent Cloud NoSQL database products.
Select Data Source Type
Click New Data Source and select the data source type, then click Next.
Configuring a Data Source: Tencent Cloud ElasticSearch
Tencent Cloud ElasticSearch data source and MongoDB data source support integration through cloud instances. The explanation will use Tencent Cloud ElasticSearch as an example, but the same applies to Tencent Cloud MongoDB data source.
Line explanation, the same applies to Tencent Cloud MongoDB data source.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Getting instance
Select the region, instance name, and ID information of the Tencent Cloud ElasticSearch instance under the account
Security Certification
Choose anonymous log in for no username and password required, or choose username and password log in and fill in the username and password information to complete security certification
Username
Username to connect to the data source
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configuring a Data Source: Redis
Redis, ElasticSearch, MongoDB, and Graph Database data sources currently only support integration via connection strings. The explanation will use Redis data source as an example, but the same applies to other data sources.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Deployment Method
Supports two deployment modes: self-built instances and public network instances. Self-built instances are deployed on Tencent CVM, while public network instances are in the client’s local IDC or other cloud resources, accessible via public network.
Region and Network
When selecting a self-built instance, you need to choose the data source's region and vpcID
IP and Port Number
Optional, enter the IP address and port number of the current data source
Account Types
If there is no username, you may leave it blank
Tencent Cloud Redis by Definition account, please provide the account
For Tencent Cloud Redis instances purchased before January 2018, please provide the instance ID
Password
Password to connect to the data source
Data Connectivity
Test whether the configured database can be connected
Create a message queue data source
The current version of Tencent WeData supports message queue data sources including Tencent Cloud CKafka, DTS-Kafka, and Kafka. CKafka and DTS-Kafka are Tencent Cloud's message queue products.
Select Data Source Type
Click New Data Source and select the data source type, then click Next.
Configure data source: Tencent Cloud CKafka
WeData supports connecting to CKafka data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Optional display name. If not filled, the data source name will be displayed
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Getting instance
Select the region, instance name, and ID information of the CKafka instance under the account
Connection Method
Tencent Cloud CKafka supports two access methods: PLAINTEXT and SASL_PLAINTEXT. For details, please see CKafka User Access Control
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configure data source: DTS-Kafka
WeData supports connecting to DTS-Kafka data sources through cloud instances.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Display name
Display name of the data source when used in the product. If not filled, the data source name will be displayed by default
Description(Optional)
Optional, description of this data source
Data Source Permission
Project sharing means all members of the current project can use the data source. 'Personal' and 'Administrator' mean only the creator and project administrators can use the data source
Cluster address
Cluster address, please go to DTS Console > Subscription Information > Intranet Address Copying
Topic
Topic name to be connected
Consumer group name
Consumer group name
Account / password
Enter account / password
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Configure data source: Kafka
WeData supports connecting to Kafka data sources through connection strings.
Parameter
Description
Data Source Name
The name of the newly created data source, defined by the user and cannot be empty. It should start with a letter and can include letters, numbers, and underscores. The length should be within 20 characters
Description(Optional)
Optional, description of this data source
Data Source Permission
Select project sharing or only for personal and admin use
Deployment Method
Supports two deployment modes: self-built instances and public network instances. Self-built instances are deployed on Tencent CVM, while public network instances are in the client’s local IDC or other cloud resources, accessible via public network.
Region and Network
When selecting a self-built instance, you need to choose the data source's region and vpcID
Kafka service list
Please enter the service list, e.g., ip1:9092, ip2:9092
Kafka Authentication Method
Supports four authentication methods: PLAINTEXT, SASL_PLAINTEXT, SASL_SSL, JAAS:
If you choose the SASL_PAINTEXT authentication method, you need to provide username and password;
If you choose the SASL_SSL authentication method, you need to provide TrustStore authentication file, TrustStore password, username, password;
If you choose the JAAS authentication method, you need to provide Kafka SASL JAAS configuration.
Data Connectivity
Test whether the configured database can be connected (if the connectivity test fails, an error message will be provided to troubleshoot the cause. The data source can still be saved, but an exception will occur when using the data source)
Appendix
Tencent Cloud MongoDB data source, Advanced parameter list supported by MongoDB data source
Parameter Type
Parameter name
Support Status
Replica set options
replicaSet
Supported
directConnection
Supported
TLS options
tls
Not supported, only supports false
ssl
Not supported, only supports false
tlsCertificateKeyFile
Not supported
tlsCertificateKeyFilePassword
Not supported
tlsCAFile
Not supported
tlsAllowInvalidCertificates
Not supported
tlsAllowInvalidHostnames
Not supported
tlsInsecure
Not supported
Timeout options
connectTimeoutMS
Supported, default 10ms
socketTimeoutMS
Supported
Compression Options
compressors
Supported
zlibCompressionLevel
Supported
Connection Pool Options
maxPoolSize
Supported
minPoolSize
Supported
maxConnecting
Supported
maxIdleTimeMS
Supported
waitQueueMultiple
Supported
waitQueueTimeoutMS
Supported
Write Concern Options
w
Supported
wtimeoutMS
Supported
journal
Supported
Read Concern Options
readConcernLevel
Supported
Read Preference Options
readPreference
Supported
maxStalenessSeconds
Supported
readPreferenceTags
Supported
Authentication Options
authSource
Supported
authMechanism
Supports CRAM-SHA-256, CRAM-SHA-1
authMechanismProperties
Not supported
gssapiServiceName
Not supported
Server Selection and Discovery Options
localThresholdMS
Supported
serverSelectionTimeoutMS
Supported
serverSelectionTryOnce
Supported
heartbeatFrequencyMS
Supported
socketCheckIntervalMS
Supported
Other Configurations
appName
Supported
retryReads
Supported
retryWrites
Supported
uuidRepresentation
Supported
loadBalanced
Supported, but requires server-side support to take effect
Yes
No
Was this page helpful?