Data Source Type | Edition |
Oracle | 11,12,19 |
sqlplus sys/password@host:port/SID AS SYSDBA;CREATE USER flinkuser IDENTIFIED BY flinkpw DEFAULT TABLESPACE LOGMINER_TBS QUOTA UNLIMITED ON LOGMINER_TBS;-- Allow users to connect to the databaseGRANT CREATE SESSION TO flinkuser;-- To support scenarios using cdb/pdb, grant users access to pdb/cdbGRANT SET CONTAINER TO flinkuser;-- Grant users SELECT privileges on all tables in the database to read dataGRANT SELECT ANY TABLE TO flinkuser;--SELECT_CATALOG_ROLE is a predefined database role that provides permissions to view database system tables. Additionally, during the incremental phase, the logminer session relies on this privilege to query the table structure.GRANT SELECT_CATALOG_ROLE TO flinkuser;--GRANT ANALYZE ANY is a grant statement used to authorize users to analyze all tables, indexes, partitions, and table functions owned by any user. When a table does not have a primary key, this privilege is used to obtain the unique index and replace the primary key with the unique index.GRANT ANALYZE ANY TO flinkuser;
sqlplus sys/password@host:port/SID AS SYSDBA
alter system set db_recovery_file_dest_size = 10G;alter system set db_recovery_file_dest = '/opt/oracle/oradata/recovery_area' scope=spfile;shutdown immediate;startup mount;alter database archivelog;alter database open;Notes: Enabling log archiving requires a database restart and will occupy disk space
-- Should now "Database log mode: Archive Mode"archive log list;
-- Enable supplemental logging for a specific table:ALTER TABLE inventory.customers ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
-- Enable supplemental logging for databaseALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
CREATE TABLESPACE logminer_tbs DATAFILE '/opt/oracle/oradata/SID/logminer_tbs.dbf' SIZE 25M REUSE AUTOEXTEND ON MAXSIZE 2048M;
--A special grant statement that grants the user the SELECT privilege on the V$DATABASE view, mainly used to check whether the database's archived log is enabled (LOG_MODE) and to get the current SCN Point of the database.GRANT SELECT ON V_$DATABASE to flinkuser;--EXECUTE_CATALOG_ROLE is a predefined role that mainly provides the capability for logminer to write the dictionary to the binlog, thus recording changes in the table structure into the logical log.GRANT EXECUTE_CATALOG_ROLE TO flinkuser;--Allow the user to read data through the LOGMINING componentGRANT LOGMINING TO flinkuser;--The EXECUTE ON DBMS_LOGMNR privilege is used to grant the user the ability to execute programs within the DBMS_LOGMNR package. The DBMS_LOGMNR package is used to analyze and extract changed data from database log files.GRANT EXECUTE ON DBMS_LOGMNR TO flinkuser;--GRANT EXECUTE ON DBMS_LOGMNR_D is a grant statement that gives the user execution privileges on the DBMS_LOGMNR_D package. DBMS_LOGMNR_D is an Oracle-provided package used to analyze online and archived redo log files at the database levelGRANT EXECUTE ON DBMS_LOGMNR_D TO flinkuser;--Logminer's data query interface is composed of a set of system-provided views. To read data from logminer, you need permissions to access the following views.GRANT SELECT ON V_$LOG TO flinkuser;GRANT SELECT ON V_$LOG_HISTORY TO flinkuser;GRANT SELECT ON V_$LOGMNR_LOGS TO flinkuser;GRANT SELECT ON V_$LOGMNR_CONTENTS TO flinkuser;GRANT SELECT ON V_$LOGMNR_PARAMETERS TO flinkuser;GRANT SELECT ON V_$LOGFILE TO flinkuser;GRANT SELECT ON V_$ARCHIVED_LOG TO flinkuser;GRANT SELECT ON V_$ARCHIVE_DEST_STATUS TO flinkuser;exit;
Parameter | Description |
Data Source | Select the Oracle data source to be synchronized |
Source Table | According to business needs, choose "All Database Tables", "Specified Table", or "Specified Database" |
| All Database Tables: Monitor all databases under the data source. New databases and tables added during the task run will be synchronized to the target by default Specified Table: Only synchronize the specified table Specified Database: Monitor the specified database and schema. Synchronize all or matching tables under the schema |
| |
| |
Read Mode | Full + Increment: Data synchronization is divided into full and increment phases. After the full phase is completed, the task enters the increment phase. The full phase will synchronize historical data in the database, and the incremental phase starts synchronizing from the binlog cdc location after the task starts. Increment: Only synchronize data from the binlog cdc position after the task starts. |
Consistency Semantics | Only represents the consistency semantics of the reading end. Supports At-least-once and Exactly-once. At-least-once: Data may be read more than once, relying on the destination end's deduplication to ensure data consistency. Suitable for scenarios with large data volumes in the full phase and using non-numeric primary keys, with high synchronization performance requirements. Exactly-once: Data is read strictly once, with some performance losses. Tables without a primary key and unique index column are not supported. Suitable for general scenarios where the source table has a numeric primary key or unique index column. The current version's two modes are state-incompatible. If the mode is changed after task submission, stateful restart is not supported. |
Advanced Settings (optional) | You can configure parameters according to business needs |
ALTER TABLE SCHEMA_NAME.TBALE_NAME ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS
Parameter | Description |
Data Source | Select the data source where the table to be synchronized is located. |
Database | Supports selection or manual input of the library name to read from. By default, the database bound to the data source is used as the default database. Other databases need to be manually entered. If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected. |
Schema | Support selection or manual input of available schema under this data source. |
Table | Supports selecting or manually entering the table name to be read. |
Read Mode | Full + Increment: Data synchronization is divided into full and increment phases. After the full phase is completed, the task enters the increment phase. The full phase will synchronize historical data in the database, and the incremental phase starts synchronizing from the binlog cdc location after the task starts. Increment Only: Only synchronize data from the binlog cdc position after the task starts. |
Consistency Semantics | Only represents the consistency semantics of the reading end. Supports At-least-once and Exactly-once. At-least-once: Data may be read more than once, relying on the destination end's deduplication to ensure data consistency. Suitable for scenarios with large data volumes in the full phase and using non-numeric primary keys, with high synchronization performance requirements. Exactly-once: Data is read strictly once, with some performance losses. Tables without a primary key and unique index column are not supported. Suitable for general scenarios where the source table has a numeric primary key or unique index column. The current version's two modes are state-incompatible. If the mode is changed after task submission, stateful restart is not supported. |
Advanced Settings (Optional) | You can configure parameters according to business needs. |
Parameter | Description |
Data Source | The Oracle data source to be written to. |
Database | Supports selection or manual input of the database name to write to By default, the database bound to the data source is used as the default database. Other databases need to be manually entered. If the data source network is not connected and the database information cannot be fetched directly, you can manually enter the database name. Data synchronization can still be performed when the Data Integration network is connected. |
Schema | Supports selecting or manually entering the Oracle data mode to be written to. |
Table | Support selection or manual entry of the table name to be written to. If the data source network is not connected and the table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected. |
Primary key | Select a field as the primary key for the target table. |
Advanced Settings (Optional) | You can configure parameters according to business needs. |
Parameter | Description |
Data Source | Select the available Oracle data source in the current project. |
Database/Table | Select the corresponding database table from this data source. |
Primary key | Select a field as the primary key for the data table. |
Advanced Settings (optional) | You can configure parameters according to business needs. |
Oracle Type | Internal Types |
NUMBER(p, s <= 0), p - s < 3 | TINYINT |
NUMBER(p, s <= 0), p - s < 5 | SMALLINT |
NUMBER(p, s <= 0), p - s < 10 | INT |
NUMBER(p, s <= 0), p - s < 19 | BIGINT |
NUMBER(p, s <= 0), 19 <= p - s <= 38 | DECIMAL(p - s, 0) |
NUMBER(p, s > 0) | DECIMAL(p, s) |
NUMBER(p, s <= 0), p - s > 38 | STRING |
FLOAT,BINARY_FLOAT | FLOAT |
DOUBLE PRECISION,BINARY_DOUBLE | DOUBLE |
NUMBER(1) | BOOLEAN |
DATE,TIMESTAMP [(p)] | TIMESTAMP [(p)] [WITHOUT TIMEZONE] |
TIMESTAMP [(p)] WITH TIME ZONE | TIMESTAMP [(p)] WITH TIME ZONE |
TIMESTAMP [(p)] WITH LOCAL TIME ZONE | TIMESTAMP_LTZ [(p)] |
CHAR(n), NCHAR(n), NVARCHAR2(n), VARCHAR(n), VARCHAR2(n), CLOB, NCLOB, XML types | STRING |
BLOB,ROWID | BYTES |
INTERVAL DAY TO SECOND,INTERVAL YEAR TO MONTH | BIGINT |
Internal Types | Oracle Type |
FLOAT | BINARY_FLOAT |
DOUBLE | BINARY_DOUBLE |
DECIMAL(p, s) | SMALLINT,FLOAT(s),DOUBLE PRECISION,REAL,NUMBER(p, s) |
DATE | DATE |
DECIMAL(20, 0) | - |
FLOAT | REAL,FLOAT4 |
DOUBLE | FLOAT8,DOUBLE PRECISION |
DECIMAL(p, s) | NUMERIC(p, s),DECIMAL(p, s) |
BOOLEAN | BOOLEAN |
DATE | DATE |
TIMESTAMP [(p)][WITHOUT TIMEZONE] | TIMESTAMP [(p)]WITHOUT TIMEZONE |
STRING | CHAR(n),VARCHAR(n),CLOB(n) |
BYTES | RAW(s),BLOB |
ARRAY | - |
Was this page helpful?