Log in to enter WeData Console > Data Development > Task Ops interface. In Task Operation and Maintenance, you can view the workflows and tasks created by the user. Find the task that needs data Replenishment through filters. Click Data Replenishment.
Note：
One-time tasks cannot Replenishment data
Note：
The task list supports multi-select tasks for bulk data Replenishment.
Steps to create a Replenishmental task
Step 1. Task Selection
Description
Parameter
Description
Replenishment Plan Name
By default, generate as patch_year and month_HHMMSS_unique number, unique within the project. For example: patch_20230624195944_548. Supports user-defined name.
Description
Users can customize the supplementary plan description, up to 256 characters (optional).
Root Task
In the workflow, select the root node for replenishment from tasks or selected tasks.
Replenishment Scope
Selected tasks: only can be selected from workflow or selected tasks.
Selected task and downstream task: can be selected from workflow or selected task and its downstream task.
Contain only downstream tasks: only can be selected from workflow or downstream tasks of selected tasks.
Note:
Note: Only tasks with status "scheduling in progress", "suspended", or "invalid" support replenishment. The number of root nodes for a single replenishment cannot exceed 500. If the limit is exceeded, please narrow down the selection range.
Select downstream task
The configuration to select downstream tasks is shown only when the replenish task scope is "selected task and downstream task" or "contain only downstream tasks".
After selecting the task, click Next.
Step 2: Parameter Configuration
Description
Parameter
Description
Instance Data Time
You can customize the data date, choose from today or earlier, with yesterday chosen by default (supports up to 10 multiple selections).
hour range
Default 00:00 - 23:59, effective for tasks with a period of hours or less. Generate replenish instances based on the data date. For example, if 01:00 - 02:00 is selected, scheduled task instances at 01:00 will be replenished, while instances at 03:00 will not.
Execution Resource Group
The default task's original resource group can be reassigned for the instance run's scheduling resource group and integration resource group, which does not affect the execution resource configuration of periodic tasks.
Check parent task
Check all upstream tasks: the selected replenishment task runs as configured dependencies.
Do not check all: the selected replenishment task does not run by task dependency and runs in parallel.
Check within selected tasks only: the selected replenishment task checks dependencies only from selected tasks, not from unselected tasks.
self-dependent
Default Task Original Self-Dependency Configuration (original task's self-dependency configuration). You can customize it and select the degree of parallelism (minimum 2, default value 2, maximum 10). After enabling parallel settings, the task's original dependency relationship will not be used.
Advanced Settings
Replenishment Instance Generation Period: You can re-specify the generation cycle of the supplemental task instances; currently, only daily instances can be converted into instances generated on the 1st of every month.
Custom parameter
Default task parameters. Users can re-specify the task's parameters to facilitate the execution of new logic by replenished instances. The new parameters will overwrite the previous task parameter values.
Custom parameters are input in the form of key-value.
Offline Synchronization Rate Limit: If a backfill task includes an offline synchronization task, the sync rate can be specified. Limit the synchronization rate based on the traffic or the number of records to reduce the read and write load at the data source or destination end. Default -1 indicates no restriction.
After configuring the time and parameters, click Next.
Step 3: Information Preview
Preview backfill includes configuration preview, Advanced Configuration Preview, and Instance Preview. Instance Preview shows the expected number of supplemented tasks and instances, as well as specific supplementary entry instances.
1. After the preview is correct, click Supplement Confirmation. If the project enables supplement approval, the replenishment plan will be generated and executed after approval.
2. After the replenishment plan is created, users can click Data Replenishment on the left side to enter the supplement list and view details and progress.
Supplement List
Data replenishment is provided in a three-level list to view and perform backfill tasks.
Primary List
Searching
Backfill task name: Retrieve backfill tasks by task name.
Node task name: Retrieve backfill tasks by task node name.
Creator: Drop-down filter. Users can choose from all members in the current project.
Creation time: Default display is the current day. Users can filter by creation time, with the latest option being the current day.
List view: By default, display supplementary task name, instance data time, task count, creation time, creator, running status, and perform.
Running Status
Running: The current backfill task has instances in running and not running states.
Completed: All instances in the current backfill task are in completed, terminated, or failed states.
Completion rate: (successful, terminated, failed) instances / total number of instances.
Success rate: (successful) instances / total number of instances.
Operation
Configuration detail: click to view the current backfill task configuration.
Terminate in batches: forcibly terminate instances in "pending execution" or "running" status. Instances in other statuses are ignored by default. Batch operation is an asynchronous operation, and instance status updates may be delayed.
Secondary List
Display current task name, task ID, responsible person, type, number of instances, status, perform.
Operation - View instance: If only one time period is selected for backfill, you can click to jump to the instance list page and select the instance for this backfill task.
Three-Level List
Display current instance data time, start time, running end time, runtime duration, status and operation.
Diagnosis: After task failure, judge the reason for failure through diagnostics and support redirection to the corresponding task for viewing details.
Operation: Support viewing logs and terminating instance operations.
