DataInLong provides the capability to read from OceanBase. This article introduces the pre-environment configuration for using OceanBase for real-time data synchronization and the current capability support.
Supported Versions
Currently, DataInLong supports real-time reading for both single tables and entire databases in OceanBase. To use the real-time reading capability, follow the version limitations below:
Data Source Type
Edition
OceanBase
Community Edition 3.1.4; Enterprise Edition 4.2.1.3
Incremental collection requires deploying logproxy; without it, the task cannot enter the incremental stage.
Logproxy must be registered with the primary account, but the data source configuration cannot use the primary account.
When the source side is OceanBase, the target end supports the use of existing databases and tables only, and does not support automatic database or table creation.
DDL changes are not supported. Schema changes at the source end will not be automatically synchronized to the target end.
Only supports At-least-once semantics and relies on the deduplication capability of the target end to ensure data uniqueness.
Node configuration for real-time whole-database reading
Data source settings
There are differences in the configuration of OceanBase MySQL mode and Oracle mode, as follows: MySQL mode configuration:
Oracle mode configuration:
Parameter
Description
Data Source Mode
Supports MySQL Compatibility Mode and Oracle Compatibility Mode
Data Source
Select the OceanBase data source to be synchronized. This will filter based on the selected data source mode. For example, if you select MySQL mode, only MySQL mode OceanBase data sources will be displayed, and Oracle mode data sources will not be shown.
Source Table
All databases and tables: Monitor all databases under the data source. Newly added databases and tables during the task run will be synchronized to the target by default.
Specific table: Under this option, you need to specify the exact table name. After setting, the task will only synchronize the specified table; if you need to add a new table for synchronization, you need to stop and restart the task.
Specific database: Under this option, you need to specify the exact database name and the table name regular expression. After setting, any new tables that match the table name expression during task execution will be synchronized to the target end by default.
Read Mode
Full + Increment: Data synchronization is divided into full and increment phases. After the full phase is completed, the task enters the increment phase. The full phase will synchronize historical data in the database, and the incremental phase starts synchronizing from the binlog cdc location after the task starts.
Increment: Synchronize data only from the binlog cdc location after the task starts.
Filter Operation
Supports Insert, Update, and Delete operations. After setting, the specified operation type data will not be synchronized. Only MySQL mode supports filtering operations.
Time Zone
Set the timezone for log timestamps. The default is Shanghai. Only MySQL mode supports timezone settings.
Advanced Settings (optional)
You can configure parameters according to business needs.
Real-time Single Table Read Node Configuration
Parameter
Description
Data Source Mode
Supports MySQL Compatibility Mode and Oracle Compatibility Mode.
Data Source
Select the data source where the table to be synchronized is located.
Database
Select the database where the table to be synchronized is located.
Table
Select the table to be synchronized.
Table Primary Key
Used to split the table into multiple shards for synchronization, applicable only in MySQL mode.
Read Mode
Full + Increment: Data synchronization is divided into full and increment phases. After the full phase is completed, the task enters the increment phase. The full phase will synchronize historical data in the database, and the incremental phase starts synchronizing from the binlog cdc location after the task starts.
Increment: Synchronize data only from the binlog cdc location after the task starts.
Advanced Settings (Optional)
You can configure parameters according to business needs.
Read Data Type Conversion Support
The supported data types and conversion mappings for OceanBase read operations are as follows (when processing OceanBase, the data types of the OceanBase data source will be mapped to the data types of the data processing engine):
OceanBase MySQL Mode Types
Internal Types
TINYINT/INT/MEDIUMINT/INTEGER
INTEGER
FLOAT,BINARY_FLOAT
FLOAT
DECIMAL/NUMETRIC
DECIMAL
DOUBLE PRECISION,BINARY_DOUBLE
DOUBLE
BOOL/BOOLEAN
BOOLEAN
DATE,TIMESTAMP [(p)]
TIMESTAMP [(p)] [WITHOUT TIMEZONE]
TIMESTAMP [(p)] WITH TIME ZONE
TIMESTAMP [(p)] WITH TIME ZONE
TIMESTAMP [(p)] WITH LOCAL TIME ZONE
TIMESTAMP_LTZ [(p)]
CHAR(n), NCHAR(n), NVARCHAR2(n), VARCHAR(n), VARCHAR2(n), CLOB, NCLOB, XML, BIT types
STRING
BLOB,ROWID
BYTES
INTERVAL DAY TO SECOND,INTERVAL YEAR TO MONTH
BIGINT
OceanBase Oracle Mode Types
Internal Types
NUMBER
DOUBLE
FLOAT,BINARY_FLOAT
FLOAT
DOUBLE PRECISION,BINARY_DOUBLE
DOUBLE
NUMBER(1)
BOOLEAN
DATE,TIMESTAMP [(p)]
TIMESTAMP [(p)] [WITHOUT TIMEZONE]
TIMESTAMP [(p)] WITH TIME ZONE
TIMESTAMP [(p)] WITH TIME ZONE
TIMESTAMP [(p)] WITH LOCAL TIME ZONE
TIMESTAMP_LTZ [(p)]
CHAR(n), NCHAR(n), NVARCHAR2(n), VARCHAR(n), VARCHAR2(n), CLOB, NCLOB, XML types
Yes
No
Was this page helpful?