Feature Overview
Code templates provide code reuse capabilities. Create a piece of code that can be used by multiple tasks to achieve code reuse and unified management capability of the code.
Application Scenario
In actual development scenarios, there are a large number of similar SQL codes with exactly the same business logic and only partial content discrepancies. For example, the input table and the output table have consistent structures but different names. At this point, developers can abstract the SQL into a template and define the differing parts as a parameter. By referencing the template in tasks and configuring different parameter values, SQL code reuse can be achieved.
Feature Description
Create Template
Enter the Offline Development > Code Template > Code Template directory, and click in the upper right corner Create Template.
Step One: Create New Code Template
Template Name: Define the definition template name.
Supported Task Types: Currently supports DLC SQL, DLC PySpark, Python task types required for selecting a template.
Template Description: Configuration template description information.
Target Folder: Define and select folders based on business needs to facilitate template management.
Step Two: Configure Template Content and Submit
Code editor: Configure the code content of the configuration template. Parameters can be defined in the form of ${}.
Right Sidebar - Template Property
Template Name: The template name can be defined or modified.
Template Description: The template description can be defined or modified.
Owner: The current operator is the default owner when creating. You can modify the owner subsequently via this entry.
Other running-related parameters: Different running configuration parameters are displayed here according to the different supported task types of the template. This parameter is valid only when the template is test-run and does not take effect in tasks that reference the template. Tasks that reference the template need to separately configure corresponding parameters. For example, DLC SQL tasks here can configure Spark SQL configuration parameters for the test-run of the current template.
Right Sidebar - Version
View detailed information of template earlier versions, perform operations such as comparison and rollback. Resubmit for the changes to take effect after rollback.
Right Sidebar - Reference Records
Support viewing tasks that refer to the current template, facilitating users' template management.
Step 3: Test-Run
Test-run. After editing the template, you can perform a test-run of the template. Click the Run operation at the top of the editor, define parameter values, and you can proceed with the template test. View the execution results and logs on the current page.
Step 4: Submit
After testing without issues, you can proceed with the submission. Click the Submit operation at the top of the editor to complete the submission. After submission, tasks referencing the template will use the code of the latest version of the template in production status.
Templates Referenced in Tasks
Step One: Create a Task
1. Enter the Offline Development > Orchestration Space > Workflow Directory, and click in the upper right corner Create Task.
2. When creating a task, if the task type to be created is DLC SQL, DLC PySpark, or Python, you can choose to create a task by referencing a template. After referencing a template, the code of the template will be displayed in the code editor of the task, and cannot be modified.
Ways to create: Select reference template
Template name: Select the template name to be referenced
Note:
The task will only refer to the code content of the template, and will not refer to other configurations such as scheduling resource groups and data sources.
Template features are only supported by three task types: DLC SQL, DLC PySpark, and Python.
Only submitted templates can be selected when creating a task.
Step Two: Configure Template Parameter Values
In Task Attribute > Scheduling Parameters, configure the values of template parameters.
You can manually fill in the parameter name and value.
You can also click auto-parse code parameters, define parameter values, and click Confirm to complete the configuration of template parameter values.
Modifying Template
For the task of referencing a template, you cannot directly modify its code. You need to uniformly modify the code in the code template. The modification method is as follows:
Open the code template and modify the code content.
Submit: In the update scenario, when submitting a code template with newly-added parameters, you can configure parameter values for the task referencing the code template during submission. The configured parameter values will be automatically filled into the corresponding task's scheduling parameters upon submission, meaning you no need to duplicate task submissions.
Note:
Upon successful submission of the code template, all tasks will reference the latest version of the code template, and the scheduling parameters configured in the submission will also take effect. However, submitting the code template will not change the task status.