- Product Introduction
- Purchase Guide
- Preparations
- Operation Guide
- Console Operation
- Project Management
- Data Integration
- Data Integration Overview
- Supported Data Sources and Read-Write Capability
- Integration resource configuration and management
- Real-time Synchronization Task Configuration and Operation and Maintenance
- Data Sources Supported by Real-time Synchronization
- Data Sources Overview
- Data Source List
- MySQL Data Source
- TDSQL-C MySQL Data Source
- Oracle Data Source
- PostgreSQL Data Source
- SQL Server Data Source
- Kafka Data Source
- DTS-Kafka Data Source
- Mongodb Data Source
- TDSQL MySQL Data Source
- OceanBase Data Source
- Dameng DM Data Source
- TiDB Data Source
- Doris Data Source
- StarRocks Data Source
- DLC Data Source
- Iceberg Data Source
- TCHouse-P Data Source
- ElasticSearch Data Source
- ClickHouse Data Source
- Greenplum Data Source
- Hive Data Source
- HBase Data Source
- TBase Data Source
- HDFS Data Source
- Whole Database Synchronization Task Configuration
- Single Table Synchronization Task Configuration
- Real-time synchronization operation and maintenance
- Data Sources Supported by Real-time Synchronization
- Offline Synchronization Task Configuration and Operation and Maintenance
- Data Sources Supported by Offline Synchronization
- Data Sources Overview
- MySQL Data Source
- TDSQL-C MySQL Data Source
- Oracle Data Source
- SQL Server Data Source
- PostgreSQL Data Source
- TDSQL for PostgreSQL Data Source
- TDSQL MySQL Data Source
- DB2 Data Source
- DaMeng DM Data Source
- OceanBase Data Source
- SAP HANA Data Source
- SAP IQ(Sybase) Data Source
- Doris/TCHouse-D Data Source
- StarRocks Data Source
- DLC Data Source
- Iceberg Data Source
- TCHouse-P Data Source
- ClickHouse Data Source
- Greenplum Data Source
- Hive Data Source
- HDFS Data Source
- HBase Data Source
- TBase Data Source
- GBase Data Source
- GaussDB Data Source
- Impala Data Source
- Kudu Data Source
- FTP Data Source
- COS Data Source
- SFTP Data Source
- REST API Data Source
- Elasticsearch/Tencent Cloud Elasticsearch Data Source
- Mongo/TencentDB Mongo Data Source
- Redis Data Source
- Kafka/CKafka Data Source
- CTSDB InfluxDB Data Source
- TDMQ Pulsar Data Source
- DataInLong Offline Synchronization Configuration and Operation and Maintenance
- Data Development：Offline Synchronization, Configuration and Ops
- Conversion Node Configuration
- Time Parameter Description
- Advanced Parameters for Offline Node
- Automatic Table Creation Capability of Synchronization Tasks
- FAQs
- Data Development
- Data Development Overview
- Engine User Guide
- Data Management
- Data Exploration
- Task Development
- Data Assets
- Data Quality
- Data Security
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Smart Ops Related Interfaces
- Data Development APIs
- DeleteProjectParamDs
- CreateWorkflowDs
- CreateTaskVersionDs
- RemoveWorkflowDs
- TriggerDsEvent
- DeleteTaskDs
- DescribeBatchOperateTask
- DeleteFilePath
- GetFileInfo
- FindAllFolder
- CreateTaskFolder
- MoveTasksToFolder
- DescribeDsFolderTree
- DeleteFile
- UploadContent
- SubmitSqlTask
- SubmitTaskTestRun
- DeleteResource
- DescribeResourceManagePathTrees
- CreateCustomFunction
- DeleteCustomFunction
- DescribeFunctionKinds
- DescribeFunctionTypes
- DescribeOrganizationalFunctions
- SaveCustomFunction
- SubmitCustomFunction
- CreateDsFolder
- DeleteDsFolder
- DescribeDsParentFolderTree
- ModifyDsFolder
- DeleteResourceFile
- DeleteResourceFiles
- Data Operations Related Interfaces
- Metadata Related Interfaces
- Task Operation and Maintenance APIs
- FreezeTasksByWorkflowIds
- DeleteWorkflowById
- DescribeDependTaskLists
- DescribeWorkflowExecuteById
- DescribeWorkflowTaskCount
- DescribeWorkflowListByProjectId
- DescribeWorkflowInfoById
- UpdateWorkflowOwner
- RunRerunScheduleInstances
- DescribeWorkflowCanvasInfo
- DescribeAllByFolderNew
- DescribeTaskRunHistory
- RunForceSucScheduleInstances
- KillScheduleInstances
- RunTasksByMultiWorkflow
- DescribeTaskByStatusReport
- DescribeStatisticInstanceStatusTrendOps
- DescribeOpsWorkflows
- DescribeSchedulerTaskTypeCnt
- BatchStopWorkflowsByIds
- DescribeInstanceByCycle
- DescribeTaskByCycleReport
- DescribeSchedulerInstanceStatus
- DescribeScheduleInstances
- CreateOpsMakePlan
- DescribeSchedulerTaskCntByStatus
- DescribeSchedulerRunTimeInstanceCntByStatus
- DescribeOpsMakePlanInstances
- DescribeOpsMakePlanTasks
- DescribeOpsMakePlans
- KillOpsMakePlanInstances
- BatchDeleteOpsTasks
- DescribeFolderWorkflowList
- DescribeTaskScript
- ModifyTaskInfo
- ModifyTaskLinks
- ModifyTaskScript
- ModifyWorkflowSchedule
- RegisterEvent
- RegisterEventListener
- SetTaskAlarmNew
- SubmitTask
- SubmitWorkflow
- TriggerEvent
- BatchModifyOpsOwners
- BatchStopOpsTasks
- CountOpsInstanceState
- DescribeDependOpsTasks
- DescribeOperateOpsTasks
- DescribeSuccessorOpsTaskInfos
- FreezeOpsTasks
- CreateTask
- ModifyWorkflowInfo
- Instance Operation and Maintenance Related Interfaces
- Data Map and Data Dictionary APIs
- Data Quality Related Interfaces
- CommitRuleGroupTask
- CreateRule
- CreateRuleTemplate
- DeleteRule
- DeleteRuleTemplate
- DescribeDataCheckStat
- DescribeDimensionScore
- DescribeExecStrategy
- DescribeQualityScore
- DescribeQualityScoreTrend
- DescribeRule
- DescribeRuleDimStat
- DescribeRuleExecDetail
- DescribeRuleExecLog
- DescribeRuleExecResults
- DescribeRuleExecStat
- DescribeRuleGroup
- DescribeRuleGroupExecResultsByPage
- DescribeRuleGroupSubscription
- DescribeRuleGroupTable
- DescribeRuleGroupsByPage
- DescribeRuleTemplate
- DescribeRuleTemplates
- DescribeRules
- DescribeRulesByPage
- DescribeTableQualityDetails
- DescribeTableScoreTrend
- DescribeTemplateDimCount
- DescribeTopTableStat
- DescribeTrendStat
- ModifyDimensionWeight
- ModifyExecStrategy
- ModifyMonitorStatus
- ModifyRule
- ModifyRuleGroupSubscription
- ModifyRuleTemplate
- DataInLong APIs
- BatchCreateIntegrationTaskAlarms
- BatchDeleteIntegrationTasks
- BatchForceSuccessIntegrationTaskInstances
- BatchKillIntegrationTaskInstances
- BatchMakeUpIntegrationTasks
- BatchRerunIntegrationTaskInstances
- BatchResumeIntegrationTasks
- BatchStartIntegrationTasks
- BatchStopIntegrationTasks
- BatchSuspendIntegrationTasks
- BatchUpdateIntegrationTasks
- CheckAlarmRegularNameExist
- CheckIntegrationNodeNameExists
- CheckIntegrationTaskNameExists
- CheckTaskNameExist
- CommitIntegrationTask
- CreateHiveTable
- CreateHiveTableByDDL
- CreateIntegrationNode
- CreateIntegrationTask
- CreateTaskAlarmRegular
- DeleteIntegrationNode
- DeleteIntegrationTask
- DeleteOfflineTask
- DeleteTaskAlarmRegular
- DescribeAlarmEvents
- DescribeAlarmReceiver
- DescribeInstanceLastLog
- DescribeInstanceList
- DescribeInstanceLog
- DescribeInstanceLogList
- DescribeIntegrationNode
- DescribeIntegrationStatistics
- DescribeIntegrationStatisticsInstanceTrend
- DescribeIntegrationStatisticsRecordsTrend
- DescribeIntegrationStatisticsTaskStatus
- DescribeIntegrationStatisticsTaskStatusTrend
- DescribeIntegrationTask
- DescribeIntegrationTasks
- DescribeIntegrationVersionNodesInfo
- DescribeOfflineTaskToken
- DescribeRealTimeTaskInstanceNodeInfo
- DescribeRealTimeTaskSpeed
- DescribeRuleTemplatesByPage
- DescribeStreamTaskLogList
- DescribeTaskAlarmRegulations
- DescribeTaskLockStatus
- DryRunDIOfflineTask
- GenHiveTableDDLSql
- GetIntegrationNodeColumnSchema
- GetOfflineDIInstanceList
- GetOfflineInstanceList
- LockIntegrationTask
- ModifyIntegrationNode
- ModifyIntegrationTask
- ModifyTaskAlarmRegular
- ModifyTaskName
- ResumeIntegrationTask
- RobAndLockIntegrationTask
- StartIntegrationTask
- StopIntegrationTask
- SuspendIntegrationTask
- TaskLog
- UnlockIntegrationTask
- CreateOfflineTask
- DescribeRealTimeTaskMetricOverview
- Platform management related APIs
- Data Source Management APIs
- Data Types
- Error Codes
- Service Level Agreements
- Related Agreement
- Contact Us
- Glossary
- Product Introduction
- Purchase Guide
- Preparations
- Operation Guide
- Console Operation
- Project Management
- Data Integration
- Data Integration Overview
- Supported Data Sources and Read-Write Capability
- Integration resource configuration and management
- Real-time Synchronization Task Configuration and Operation and Maintenance
- Data Sources Supported by Real-time Synchronization
- Data Sources Overview
- Data Source List
- MySQL Data Source
- TDSQL-C MySQL Data Source
- Oracle Data Source
- PostgreSQL Data Source
- SQL Server Data Source
- Kafka Data Source
- DTS-Kafka Data Source
- Mongodb Data Source
- TDSQL MySQL Data Source
- OceanBase Data Source
- Dameng DM Data Source
- TiDB Data Source
- Doris Data Source
- StarRocks Data Source
- DLC Data Source
- Iceberg Data Source
- TCHouse-P Data Source
- ElasticSearch Data Source
- ClickHouse Data Source
- Greenplum Data Source
- Hive Data Source
- HBase Data Source
- TBase Data Source
- HDFS Data Source
- Whole Database Synchronization Task Configuration
- Single Table Synchronization Task Configuration
- Real-time synchronization operation and maintenance
- Data Sources Supported by Real-time Synchronization
- Offline Synchronization Task Configuration and Operation and Maintenance
- Data Sources Supported by Offline Synchronization
- Data Sources Overview
- MySQL Data Source
- TDSQL-C MySQL Data Source
- Oracle Data Source
- SQL Server Data Source
- PostgreSQL Data Source
- TDSQL for PostgreSQL Data Source
- TDSQL MySQL Data Source
- DB2 Data Source
- DaMeng DM Data Source
- OceanBase Data Source
- SAP HANA Data Source
- SAP IQ(Sybase) Data Source
- Doris/TCHouse-D Data Source
- StarRocks Data Source
- DLC Data Source
- Iceberg Data Source
- TCHouse-P Data Source
- ClickHouse Data Source
- Greenplum Data Source
- Hive Data Source
- HDFS Data Source
- HBase Data Source
- TBase Data Source
- GBase Data Source
- GaussDB Data Source
- Impala Data Source
- Kudu Data Source
- FTP Data Source
- COS Data Source
- SFTP Data Source
- REST API Data Source
- Elasticsearch/Tencent Cloud Elasticsearch Data Source
- Mongo/TencentDB Mongo Data Source
- Redis Data Source
- Kafka/CKafka Data Source
- CTSDB InfluxDB Data Source
- TDMQ Pulsar Data Source
- DataInLong Offline Synchronization Configuration and Operation and Maintenance
- Data Development：Offline Synchronization, Configuration and Ops
- Conversion Node Configuration
- Time Parameter Description
- Advanced Parameters for Offline Node
- Automatic Table Creation Capability of Synchronization Tasks
- FAQs
- Data Development
- Data Development Overview
- Engine User Guide
- Data Management
- Data Exploration
- Task Development
- Data Assets
- Data Quality
- Data Security
- API Documentation
- History
- Introduction
- API Category
- Making API Requests
- Smart Ops Related Interfaces
- Data Development APIs
- DeleteProjectParamDs
- CreateWorkflowDs
- CreateTaskVersionDs
- RemoveWorkflowDs
- TriggerDsEvent
- DeleteTaskDs
- DescribeBatchOperateTask
- DeleteFilePath
- GetFileInfo
- FindAllFolder
- CreateTaskFolder
- MoveTasksToFolder
- DescribeDsFolderTree
- DeleteFile
- UploadContent
- SubmitSqlTask
- SubmitTaskTestRun
- DeleteResource
- DescribeResourceManagePathTrees
- CreateCustomFunction
- DeleteCustomFunction
- DescribeFunctionKinds
- DescribeFunctionTypes
- DescribeOrganizationalFunctions
- SaveCustomFunction
- SubmitCustomFunction
- CreateDsFolder
- DeleteDsFolder
- DescribeDsParentFolderTree
- ModifyDsFolder
- DeleteResourceFile
- DeleteResourceFiles
- Data Operations Related Interfaces
- Metadata Related Interfaces
- Task Operation and Maintenance APIs
- FreezeTasksByWorkflowIds
- DeleteWorkflowById
- DescribeDependTaskLists
- DescribeWorkflowExecuteById
- DescribeWorkflowTaskCount
- DescribeWorkflowListByProjectId
- DescribeWorkflowInfoById
- UpdateWorkflowOwner
- RunRerunScheduleInstances
- DescribeWorkflowCanvasInfo
- DescribeAllByFolderNew
- DescribeTaskRunHistory
- RunForceSucScheduleInstances
- KillScheduleInstances
- RunTasksByMultiWorkflow
- DescribeTaskByStatusReport
- DescribeStatisticInstanceStatusTrendOps
- DescribeOpsWorkflows
- DescribeSchedulerTaskTypeCnt
- BatchStopWorkflowsByIds
- DescribeInstanceByCycle
- DescribeTaskByCycleReport
- DescribeSchedulerInstanceStatus
- DescribeScheduleInstances
- CreateOpsMakePlan
- DescribeSchedulerTaskCntByStatus
- DescribeSchedulerRunTimeInstanceCntByStatus
- DescribeOpsMakePlanInstances
- DescribeOpsMakePlanTasks
- DescribeOpsMakePlans
- KillOpsMakePlanInstances
- BatchDeleteOpsTasks
- DescribeFolderWorkflowList
- DescribeTaskScript
- ModifyTaskInfo
- ModifyTaskLinks
- ModifyTaskScript
- ModifyWorkflowSchedule
- RegisterEvent
- RegisterEventListener
- SetTaskAlarmNew
- SubmitTask
- SubmitWorkflow
- TriggerEvent
- BatchModifyOpsOwners
- BatchStopOpsTasks
- CountOpsInstanceState
- DescribeDependOpsTasks
- DescribeOperateOpsTasks
- DescribeSuccessorOpsTaskInfos
- FreezeOpsTasks
- CreateTask
- ModifyWorkflowInfo
- Instance Operation and Maintenance Related Interfaces
- Data Map and Data Dictionary APIs
- Data Quality Related Interfaces
- CommitRuleGroupTask
- CreateRule
- CreateRuleTemplate
- DeleteRule
- DeleteRuleTemplate
- DescribeDataCheckStat
- DescribeDimensionScore
- DescribeExecStrategy
- DescribeQualityScore
- DescribeQualityScoreTrend
- DescribeRule
- DescribeRuleDimStat
- DescribeRuleExecDetail
- DescribeRuleExecLog
- DescribeRuleExecResults
- DescribeRuleExecStat
- DescribeRuleGroup
- DescribeRuleGroupExecResultsByPage
- DescribeRuleGroupSubscription
- DescribeRuleGroupTable
- DescribeRuleGroupsByPage
- DescribeRuleTemplate
- DescribeRuleTemplates
- DescribeRules
- DescribeRulesByPage
- DescribeTableQualityDetails
- DescribeTableScoreTrend
- DescribeTemplateDimCount
- DescribeTopTableStat
- DescribeTrendStat
- ModifyDimensionWeight
- ModifyExecStrategy
- ModifyMonitorStatus
- ModifyRule
- ModifyRuleGroupSubscription
- ModifyRuleTemplate
- DataInLong APIs
- BatchCreateIntegrationTaskAlarms
- BatchDeleteIntegrationTasks
- BatchForceSuccessIntegrationTaskInstances
- BatchKillIntegrationTaskInstances
- BatchMakeUpIntegrationTasks
- BatchRerunIntegrationTaskInstances
- BatchResumeIntegrationTasks
- BatchStartIntegrationTasks
- BatchStopIntegrationTasks
- BatchSuspendIntegrationTasks
- BatchUpdateIntegrationTasks
- CheckAlarmRegularNameExist
- CheckIntegrationNodeNameExists
- CheckIntegrationTaskNameExists
- CheckTaskNameExist
- CommitIntegrationTask
- CreateHiveTable
- CreateHiveTableByDDL
- CreateIntegrationNode
- CreateIntegrationTask
- CreateTaskAlarmRegular
- DeleteIntegrationNode
- DeleteIntegrationTask
- DeleteOfflineTask
- DeleteTaskAlarmRegular
- DescribeAlarmEvents
- DescribeAlarmReceiver
- DescribeInstanceLastLog
- DescribeInstanceList
- DescribeInstanceLog
- DescribeInstanceLogList
- DescribeIntegrationNode
- DescribeIntegrationStatistics
- DescribeIntegrationStatisticsInstanceTrend
- DescribeIntegrationStatisticsRecordsTrend
- DescribeIntegrationStatisticsTaskStatus
- DescribeIntegrationStatisticsTaskStatusTrend
- DescribeIntegrationTask
- DescribeIntegrationTasks
- DescribeIntegrationVersionNodesInfo
- DescribeOfflineTaskToken
- DescribeRealTimeTaskInstanceNodeInfo
- DescribeRealTimeTaskSpeed
- DescribeRuleTemplatesByPage
- DescribeStreamTaskLogList
- DescribeTaskAlarmRegulations
- DescribeTaskLockStatus
- DryRunDIOfflineTask
- GenHiveTableDDLSql
- GetIntegrationNodeColumnSchema
- GetOfflineDIInstanceList
- GetOfflineInstanceList
- LockIntegrationTask
- ModifyIntegrationNode
- ModifyIntegrationTask
- ModifyTaskAlarmRegular
- ModifyTaskName
- ResumeIntegrationTask
- RobAndLockIntegrationTask
- StartIntegrationTask
- StopIntegrationTask
- SuspendIntegrationTask
- TaskLog
- UnlockIntegrationTask
- CreateOfflineTask
- DescribeRealTimeTaskMetricOverview
- Platform management related APIs
- Data Source Management APIs
- Data Types
- Error Codes
- Service Level Agreements
- Related Agreement
- Contact Us
- Glossary
HDFS Single Table Write Node Configuration
1. On the Data Integration page, in the left sidebar, click Real-time Sync.
2. At the top of the Real-time Sync page, select Single Table Sync to create a new one (you can choose between form and canvas modes) and enter the configuration page.
3. Click the left write, click and select HDFS node and configure node information.
4. You can refer to the table below to configure parameter information.
Parameters | Description |
Data Destination | Select available HDFS data sources in the current project. |
File Path | Path information of the file system. The path supports using '*' as a wildcard. After specifying the wildcard, multiple file information will be traversed. |
File Type | HDFS supports four file types: txt, orc, parquet, csv. txt: represents TextFile file format. orc: represents ORCFile file format. parquet: represents standard Parquet file format. csv: represents standard HDFS file format (logical two-dimensional table). |
Advanced Settings | You can configure parameters according to business needs. |
5. Preview Data Fields and map them with Read Node Configuration Fields, click Save.
HDFS Log Collection Write Node Configuration
Parameters | Description |
Data Destination | Select available HDFS data sources in the current project. |
File Path | Path information of the file system. The path supports using '*' as a wildcard. After specifying the wildcard, multiple file information will be traversed. |
File Type | HDFS supports four file types: txt, orc, parquet, csv. |
Advanced Settings (optional) | You can configure parameters according to business needs. |
Yes
No
Was this page helpful?