tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Cloud WeData
    Documentation
    Download PDF

    GBase Data Source

    Last updated: 2024-11-01 17:52:37
    Download PDF

      GBase Offline Single Table Read Node Configuration

      
      
      
      Parameters
      Description
      Data Source
      Available GBase Data Sources.
      Database
      Supports selection or manual input of the library name to read from.
      By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
      If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
      Schema
      Supports selection or manual input of the Schema name to be read.
      Table
      Supports selection or manual input of the table name to be read. Supports simultaneous reading of multiple tables. When configured for multiple tables, you need to ensure that the schema structure of the tables is consistent. GBase does not check the logical consistency of the tables.
      Split Key
      When extracting data with GBase, if splitPk is specified, it means you wish to sharding data using the field represented by splitPk. As a result, concurrent tasks for Data Synchronization will be initiated to improve the efficiency of Data Synchronization.
      It is recommended to use the table primary key as splitPk because it is usually evenly distributed, thus the resulting partitions are less likely to have data hotspots.
      Currently, splitPk only supports partitioning integer data, not strings, floating points, or dates. If you specify an unsupported type, the splitPk feature will be ignored, and single-channel synchronization will be used.
      If the splitPk value is set to empty, it is considered that partitioning for a single table is not allowed, so single-channel extraction will be used.
      Filter Conditions (Optional)
      Fill in the corresponding filter statements according to the data type. This statement will be used as the filter condition for the data to be synchronized. GBase splices SQL according to the specified where condition and extracts data based on that SQL. For example, during testing, you can specify the where condition as limit 10. In actual business scenarios, you would typically select the data of the current day for synchronization, specifying the where condition as gmt_create > $bizdate.
      The where condition can effectively perform incremental business synchronization.
      If the where condition is empty, it is considered that the entire table's information will be synchronized.

      GBase Offline Single Table Write Node Configuration

      
      
      
      Parameters
      Description
      Data Destination
      GBase Data Source to be written.
      Database
      Supports selection or manual input of the database name to write to
      By default, the database bound to the data source is used as the default database. Other databases need to be manually entered.
      If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected.
      Schema
      Supports selection or manual input of the Schema name to be read.
      Table
      Supports selection or manual input of the table name to write to
      If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
      When the source table is of Oracle type, GBase supports one-click creation of the target table.
      Whether to Clear Table
      Before writing to the GBase data table, you can manually choose whether to clear the data table.
      Batch Submission Size
      The record size for Batch Submission can greatly reduce the Number of Network Interactions between the Data Synchronization system and GBase, thereby improving overall Throughput. If this value is set too high, it may cause OOM exceptions in the Data Synchronization process.
      Pre-Executed SQL (Optional)
      The SQL statement executed before the synchronization task. Fill in the correct SQL syntax according to the data source type, such as clearing the old data in the table before execution (truncate table tablename).
      Post-Executed SQL (Optional)
      The SQL statement executed after the synchronization task. Fill in the correct SQL syntax according to the data source type, such as adding a timestamp (alter table tablename add colname timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP).

      Data type conversion support

      Read

      GBase Data Type
      Internal Types
      TINYINT,SMALLINT,INTEGER,MEDIUMINT,BIGINT
      Long
      DECIMAL,FLOAT,DOUBLE,REAL,NUMERIC
      Double
      CHAR,VARCHAR,TINYTEXT,TEXT,MEDIUMTEXT,LONGTEXT
      String
      DATE,TEAR,DATETIME,TIMESTAMP
      Date
      TINYBLOB,MEDIUMBOLB,LONGBLOB
      Bytes
      BOOLEAN
      Boolean

      Write

      Internal Types
      GBase Data Type
      Long
      TINYINT,SMALLINT,INTEGER,MEDIUMINT,BIGINT
      Double
      DECIMAL,FLOAT,DOUBLE,NUMERIC,REAL
      String
      CHAR,VARCHAR,TINYTEXT,TEXT,MEDIUMTEXT,LONGTEXT
      Date
      DATE,YEAR,TIMESTAMP,DATETIME
      Bytes
      TINYBLOB,MEDIUMBOLB,LONGBLOB
      BOOLEAN
      Boolean
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy