tencent cloud

All product documents
Stream Compute Service
DocumentationStream Compute ServiceGetting StartedCreating a Python Job
Creating a Python Job
Download PDF
Last updated: 2023-11-08 10:52:05
Creating a Python Job
Last updated: 2023-11-08 10:52:05
Download PDF

Prerequisites

Python jobs must be executed in a private cluster. If you don't have one yet, see Creating a Private Cluster. Currently, Stream Compute Service supports running Python jobs developed using the open-source framework Flink 1.13 in environments installed with Python 3.7.

Step 1. Prepare your Python package

You need to develop your own Python package in order to create a Python job.

Step 2. Upload the Python package

‍Log in to the Stream Compute Service console‍. Select Dependencies on the left sidebar and click Create dependency. Select the same region as the private cluster. For the upload method, Local is selected by default. Click Select and, in the window that pops up, select and upload the Python package in step 1. Enter a package name, package description, and version description and click Confirm. The package will appear in the package list.

Step 3. Create a Python job

Log in to the Stream Compute Service console. ‍Enter a workspace and click Jobs on the left menu. On the Jobs page, click Create job, select Python job as the type, enter a job name, select a running cluster (where the job will be executed), and click Confirm.

Step 4. Grant access to related services

Select the job created and click Development & Testing. If you haven't granted Stream Compute Service access to related services, in the window that pops up, click Authorize now to allow Stream Compute Service to access services such as CKafka and TencentDB.

Step 5. Configure the Python job

In Development & ‍Testing, select the file you uploaded from the drop-down list of Main package and enter the ‍entry class and class parameters. Stream Compute Service is currently built in with Python 3.7, which you can select as the Python environment. ‍You can click Job parameters to ‍modify the default operator parallelism and add external dependencies.

Step 6. Publish and run the Python job

Click Publish draft to run and check the job and then click Confirm to publish the job. A version of the job will be created, and a version number will be automatically generated. Under Manage version, you can view and switch to different versions of the job. Select the version you want to run, click Run version, and then click Confirm to run the job.

Step 7. View job execution details

After you click Confirm, the job status will become "Operating" first and change to "Running" after the job is successfully run. After the job is successfully run, you can view its execution details via Monitoring, Log, and Flink UI.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon