On the selected project page, select Add in the application column, select application type as WDL, enter the necessary information (application name, type, etc.), and submit to create a new application.
The platform supports multiple ways to create new WDL applications, including importing code repository files through git or directly importing public applications provided by the platform. It also supports creating blank applications and building application files from 0 to 1 through coding mode or graphical edditing mode
Method One: Creating a Application by Importing a Git URL
1. Fill in the basic application information (such as application name and description) and git repository-related information. For private repositories, you need to fill in the git username and password (if a Github address is involved, fill in the token).
2. Click Next, verify the git information, select the target branch or tag, and confirm to import.
Method Two: Create Blank Application
Creat a WDL application as follows: In the Create Application pop-up, select Create a blank application, fill in necessary application information, such as application name, edit mode, and description, and complete the addition after confirmation.
Method Three: Import Public Application
Import Method One: Select Public Application in the Public Repository
Select Public Repository from the homepage sidebar, select a tab on the top to switch to Public Application, find the public application you need by searching the name or filtering the category, and click Import into the project on the application card.
Then, the Import Project pop-up appears. Next, enter the application name and select the project to be imported (public applications are currently only supported in coding mode).
Import Method Two: Import Public Application in the project
Enter any project and click Import Public Applications in the upper right corner of the application module.
The dropdown list displays commonly used public applications. It can be quickly imported into the current project. Click More Applications below the dropdown list to jump to the Public Repository > Public Application List page to view more public applications.
Click Import project of any public application in the dropdown list, and a pop-up for importing the project will be displayed. Fill in the application name. (Currently, public applications can only be used in coding mode.)
Note:
Application names cannot be repeated in the same project.
Edit Application
1. Import WDL file
In the resource manager of the application edit page, you can upload a local WDL file.
WDL files must end with ".wdl".
Uploading a zip file can pack and upload local projects. If the uploaded zip file is large, wait for a while after the upload is successful and refresh the page to view the file.
In the resource manager, you can use the right-click menu to add, delete, rename, and set the main file.
The main file needs to be saved to take effect.
2. Edit WDL File
Click the .wdl file created in the resource manager to open the editing interface, where you can edit the WDL file.
An application must have one and only one workflow as the main file of the application, such as main.wdl.
Write WDL workflows according to the WDL syntax specification. For the complete WDL syntax, see WDL official documentation.
The DAG overview diagram can help sort out the dependencies between tasks. Click the DAG button in the upper right corner to view the current application overview diagram. Currently, only the DAG overview diagram of a single WDL application is supported.
3. Save and Verify
Click Save and Verify to save and verify the file, and you need to select the WDL main file as the entrance.
Do not exit the browser before saving the file to avoid losing the edited file content.
Workflow meta supports specifying the following fields to set the final output path to COS:
○ final_workflow_outputs_dir: COS path for running results output.
○ use_relative_output_paths: Use relative path output; that is, only keep the directory level within the task running directory. When you use it, note that different tasks need to specify different directories to avoid COS files overwriting each other.
Run Applications
There are two options for running the application. One is Run at the bottom of the application on the project page, and the other is Run Application at the top-right corner of the application's edit page.
During the running, you need to select the WDL file that serves as the run entrance as the main file.
1. Set up the task
Task names cannot be the same within the same project
Running Options
Use Call-Caching feature: When the Call-Caching feature is used, the same task does not need to be executed repeatedly, and the previous calculation results will be automatically used.
Use relative directory output: If relative directory output is used, all job output files will be archived in the same directory; that is, only the directory level within the task running directory is retained, and it is necessary to ensure that the output files of each job have different names. If it is not checked, all results of multiple task runs will be retained in the specified directory.
Example:
When the output directory of a workflow task is set to cos://bucket-12345678/pipeline/output, the output file in the job is sample.vcf:
If you select Use relative directory output, the final output file is : cos://bucket-12345678/pipeline/output/sample. If there are output files with the same name in different jobs, a file conflict will occur in the task output directory, and the conflicting files will overwrite each other;
If you do not select the Use relative directory output option, the final output file is: cos://bucket-12345678/pipeline/output/wgs/ade68a6d876e8d-8a98d7e9-ad989a8d/call-gatk/execution/sample.vcf. Different jobs are output to different directories, and duplicate output files will not cause conflicts.
The platform supports two running failure modes (Cromwell features), i.e. NoNewCalls and ContinueWhilePossible. The appropriate operations to take in case of job failure during workflow execution can be specified.
NoNewCalls (default): Once a job fails, Cromwell will not immediately start any new programs. Cromwell will still monitor the remaining jobs until they are completed (successfully or not).
ContinueWhilePossible: Try to run as many jobs as possible until no more can be started. When all running jobs are completed, the workflow will fail.
Output Directory (optional): When you specify a task output directory, the job output files will be archived in that directory.
Cache Retention Period: The cache retention period is divided into 4 levels: 24 hours, 3 days, 7 days, and permanent.
Note:
The cache will be deleted when it is beyond the set retention period. This may affect the call caching feature.
2. Set running parameters
The running parameters can be filled in according to the parameters parsed by WDL.
Running parameters must provide valid values according to the type declared in WDL.
Some parameters that have default values can retain their default values.
Variable names with an asterisk in the upper left corner are required inputs.
You can save the running parameters as a template and use it directly later, or you can download the input parameters as a JSON file.
The platform provides reference templates for public applications. When you import a project, match the parameter template of the region where the project is located. You can select the corresponding template on the running parameter setting page.
3. Submit a single running There is no need to upload a table for a single running, and manually completing the setting of running parameters is sufficient to complete the submission.
4. Batch Execution If the input files are different but the overall process is the same, users can also batch-submit tasks by uploading a table. For details, see Visual Running Groups Submission and Management.
View Tasks
1. View task group status
In the project, the running progress and time of eachtask group are displayed.
In the task batch column, each row of data corresponds to the status of all task groups submitted at one time. The corresponding status shows the summary of the status of all task groups. The hovering task status column will display the specific progress.
A single running submission is also considered a task batch.
2. Viewing task details
Select the Details option for a task group to enter its details page and view the execution progress of that batch.
On the batch details page, you can view the execution progress of task groups.
You can view the status of a single subtask in the running list. The subtask name displayed in the subtask column is the corresponding subtask_id.
The job list displays all jobs in a running submission. They are divided into three types, i.e. task, scatter, and workflow. Scatter and workflow contain sub-projects. The sub-projects can be viewed when clicked.
The task is the smallest task unit. You can view command lines, resources, logs, and monitoring in job details.
Version Management
The version management feature covers two major features:publishing and historical timeline. It can help you better understand the progress and quality of the application, improving the efficiency, quality, and performance of the application and optimizing the development and management of workflows.
Publish feature: Through the publish feature, you can easily publish the official version. After the permission management feature is launched, you can control the viewing and usage permissions of different members to better manage the version of the application.
Historical Timeline: Through the timeline, you can view the historical and version records and trace the application development process. This can help you promptly discover, solve, and optimize problems, improve application quality and performance, and optimize your development process and management standards.
Publishing Application Version
After the workflow phase development is completed, you can publish the official version. Before publishing, confirm that the application has no syntax errors and the main file has been set. Click Publish in the upper right corner before confirming that everything is correct, and you can enter the publishing process.
Clicking Publish will trigger the save and verification process. After passing the verification, the publishing process will begin. A pop-up of the published version will appear. Fill in the published version name and description, select the parameter template, and complete the publishing after confirmation.
After the publishing is completed, you can switch to this version in the historical version timeline. Viewing and running this historical version is supported.
Note:
Historical versions can only be viewed and run but cannot be edited, deleted, or published within the application.
Viewing Historical and Version Records
View Timeline
The timeline is a visual display of the complete history of the application, including version records and historical records. The timeline view is arranged in chronological order and shows all updates to the application workflow files. You can use the timeline to trace the key time nodes of the development process and compare different versions and historical records, making it easier to manage and control the development process.
View Historical Records
In the timeline view, each time an application file is updated and saved, a historical record is generated. Click any historical record to switch the content of the resource manager interface to the files corresponding to the historical record.
Note:
The historical records are for viewing only and cannot be edited, run, published, or deleted.
View Historical Versions
In the timeline view, the officially published versions are also arranged in chronological order according to the historical versions. Click any historical version, and the resource manager interface content will switch to the files corresponding to the historical version.
Copy Application
The platform supports quick copying of the application replica within a project or across projects. If you need to copy an application, click the application card extension option and select Copy;
In the Copy a Replica pop-up, fill in the application name and description, select the application version, target project, and parameter template (optional), and complete the copy after confirmation. You can view, edit, or run the application in the target project.
Yes
No
Was this page helpful?