Stream Compute Service
- Releases Notes and Announcements
- Getting Started
- Operation Guide
- Managing Jobs
- Developing Jobs
- Monitoring Jobs
- Events and Diagnosis
- Managing Metadata
- Tuning Jobs
- Managing Clusters
- Managing Permissions
- SQL Developer Guide
- DDL Statements
- Connectors
- SET Statement
- Operators and Built-in Functions
- Identifiers and Reserved Words
- ETL Developer Guide
Custom Connectors
Last updated: 2023-11-08 14:25:19
Overview
If the built-in connectors cannot meet your needs, you can customize your own connectors by uploading source and sink API classes and dynamically loading and calling them when the job is executed.
Versions
Flink Version | Description |
1.11 | Supported |
1.13 | Supported |
1.14 | Supported |
1.16 | Supported |
Prepare your connector package
You can either use a third-party connector package (such as Bahir) or build your own package.
Apache Bahir
Apache Bahir offers common source and sink extensions for Flink.
Currently, the following third-party components are provided:
Building your own package
Build and upload the connector package
Step 1. Compile the source code
We recommend you refer to existing connector projects to modify the
pom.xml file to import dependencies and create a JAR package in Maven.Note
We recommend you use
maven-shade-plugin to shade common dependencies such as Apache Commons and Guava to avoid conflicts between the imported dependencies and classes of Stream Compute Service.Step 2. Upload the package
Go to the Dependencies page of the Stream Compute Service console and upload the connector package. For the first upload, the package will be v1. If you upload a new version, it will be v2, and so on.
### Step 3. Reference the package
On the details page of your job, under the Job parameters tab, select the package version you uploaded.
Note
To avoid unexpected errors, make sure you select the correct package version.
### Step 4. Save/Publish the job
Click Save to save the settings or Publish draft to publish the job.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No