Log in to enter WeData Console > Data Development > Task Ops interface. In Task Operation and Maintenance, you can view the workflows and tasks created by the user. Find the task that needs data Replenishment through filters. Click Data Replenishment.
Note:
One-time tasks cannot Replenishment data
Note:
The task list supports multi-select tasks for bulk data Replenishment.
Steps to create a Replenishmental task
Step 1. Task Selection
Feature Description:
Parameter
Description
Replenishment Plan Name
The default name is generated using patch_YYYYMMDDHHmmss_UniqueNumber. It is unique within the project. For example: patch_20230624195944_548. User-defined names are also supported.
Description
Users can define a supplementary plan description up to 256 characters. Optional.
Root Task
Select the root node that needs supplementation within the workflow or from selected tasks.
Replenishment Scope
Selected Tasks: Choose only within the workflow or from selected tasks.
Selected Tasks and Downstream Tasks: Choose within the workflow or from selected tasks and their downstream tasks.
Only Downstream Tasks: Choose only from the downstream tasks in the workflow or from selected tasks.
Note:
Only tasks with the status "In Scheduling", "Paused", or "Invalid" support backfilling. A single backfill cannot exceed 500 root nodes; if this limit is exceeded, users should narrow down their selection scope.
Select Downstream Tasks
The configuration for selecting downstream tasks is displayed only when the scope of the backfill task is chosen as "Selected tasks and downstream tasks" or "Only downstream tasks".
After selecting the task, click Next
Step 2: Parameter Configuration
Feature Description:
Parameter
Description
Instance Data Time
Users can define the data date. The selectable range is today and earlier, with yesterday selected by default (up to 10 time periods can be selected).
Hour Range
Default is 00:00 - 23:59. Effective only for tasks with an hourly or shorter cycle. The final supplementary instance is generated based on the data date. For example, if 01:00 - 02:00 is selected, the hourly task scheduled at 01:00 will be supplemented, but the task at 03:00 will not be supplemented.
Execution Resource Group
Default resources group of the original task. The user can reassign the execution resource group and integration resource group for the supplement instance, without affecting the periodic task execution resources configuration.
Check Parent Task
Check all upstream tasks: The selected supplement tasks will run according to their configured dependencies.
Check none: The selected supplement tasks will not follow task dependencies and will run in parallel.
Check only in selected tasks: The selected supplement tasks will only check dependencies within the selected tasks, not from non-selected tasks.
Self-Dependency
Default self-dependency configuration of the original task (the original task's self-dependency configuration). The user can self-define and choose the parallelism (minimum 2, default 2, maximum 10). Setting parallelism will override the task's original self-dependency configuration.
Advanced Settings
Supplemental Instance Generation Cycle: You can re-specify the generation cycle of the supplemental task instances; currently, only daily instances can be converted into instances generated on the 1st of every month.
Custom Parameters:
Default task parameters can be re-specified by the user to facilitate the execution of new logic in supplemental instances. The new parameters will override the previous task parameter values.
Custom parameters should be input in key-value format.
Offline Synchronization Rate Limit: If the supplementary tasks include offline synchronization tasks, a sync rate can be specified. The rate is limited based on traffic or the number of records to protect the read and write pressure of the data source or destination. Default -1 means no rate limit.
After configuring the time and parameters, click Next.
Step 3: Information Preview
The preview includes Configuration Preview, Advanced Configuration Preview, and Instance Preview. The Instance Preview shows the expected number of supplemented tasks and instances, as well as the specific supplementary instance.
After verifying the preview, click Confirm Supplementation to complete the creation of the supplemental task. You can then click the left-side Data Supplementation to view the supplementation list.
Supplementation List
Data supplementation is provided in a three-level list format for viewing and operating supplementation tasks.
First-level List
Search
Supplementary Task Name: Search for supplementation tasks by name.
Node Task Name: Search for supplementation tasks by the names of task nodes included in the supplementary task.
Creator: Drop-down filter; users can select any member of the current project.
Creation Time: By default, displays the current day. Users can filter by creation time, with the latest option being the current day.
List Display: By default, it shows task name, instance data time, task count, creation time, creator, running status, operations.
Running Status
Running: Instances in the current supplementary task are in running status, with some not yet started.
Completed: All instances in the current supplementary task are either completed, terminated, or failed.
Completion Rate: Total number of instances with statuses (success, terminated, failed) / total number of instances.
Success Rate: Total number of instances with status (success) / total number of instances.
Operation
Configuration Details: Click to view the configuration details of the current supplementary task.
Batch Stop: Force stop instances with status "Pending Execution" or "Running"; other statuses are ignored by default! Batch operations are asynchronous, and instance statuses may be delayed.
Secondary List
Display current task name, task ID, person in charge, type, number of instances, status, operations.
Operation - View Instance: If only one time period is selected for the data fill, you can click to jump to the instance list page and select the instances of this task for this data fill.
Tertiary List
Display current instance data time, start time of operation, end time of operation, runtime duration, status, and operations.
Diagnosis: When a task fails, the diagnosis can be used to determine the failure's cause, and it supports redirecting to the corresponding task details for review.
Operation: Support viewing logs and terminating instances.
Yes
No
Was this page helpful?