Data Lake Compute (DLC) supports configuring network (VPC) for data engine, facilitating management of engine access to different data source networks.
Network Configuration Type
According to different business scenarios, DLC provides two network configuration types.
Enhanced network configuration: suitable for accessing the data under one VPC with high speed and stability.
Note:
A data engine of a non-Spark job type can only be bound to one enhanced network configuration.
Cross-origin network configuration: suitable for cross-origin federated data query that needs to access multiple VPCs. A data engine can support binding multiple cross-origin network configurations.
Network Configuration Status
Initializing: The network configuration is being initialized. At this point, the network is not active.
Success: The network configuration takes effect on the bound engine.
Failure: The network configuration fails and can be deleted and reconfigured.
Network Configuration Security Policy
If you have configured a security group policy for the VPC, you need to add inbound rules for different network configuration types.
Enhanced network: Add inbound rules for the IP range of the VPC where the data source is located to the security group.
Cross-source network: Add inbound rules for the IP range of the engine bound to the network configuration to the security group.
Create Network Configuration
1. Log in to the DLC console and choose service region. 2. Enter Resource Management > Network Connection Configuration through the left sidebar.
3. Click the Create Network Configuration button to enter the Create Configuration page.
The configuration parameters are as follows:
|
Network Configuration Type | Yes | Select according to the use case Enhanced network configuration: suitable for data scenarios that require high-speed and stable access to a VPC. Cross-origin network configuration: suitable for cross-origin federated query analysis scenarios that need to access data under multiple VPCs. |
Configuration Name | Yes | Supports Chinese, English, and _, with a number of characters not more than 35. |
Instance source | Yes | Two sources are supported: Data catalog of DLC: Option the data catalog that has created a connection in the data management of DLC currently New network configuration: Select a new data source to create a network connection. Currently, the data source supports MySQL, Kafka, EMR HDFS (COS, HDFS, Chdfs), Postgresql, SqlServer, Clickhouse. If the data source associated with the network configuration to be created is not yet supported, you can select another option and manually specify a VPC. |
Catalog | Yes | Select the corresponding data catalog according to the source of the selected instance. The range of selectable data catalogs will be related to your account permission. |
Data source VPC | No | The data engine network will connect all subnets in the VPC. |
Bound data engine | Yes | Select the data engine associated with this network configuration. If the data engine is in isolated or initializing status, it will be unable to select. |
Configuration Description | No | Not more than 100 characters. |
4. Fill in, complete the settings and save. Then you can create a network configuration.
Note:
Once created, the network is in the initialization state. Subsequently, you can view the status in the list.
Delete Network Configuration
You can perform a deletion operation to manage the deletion of network configurations that are no longer needed or have failed to configure. Directions:
2. Enter Engine Management > Engine Network Configuration through the left sidebar.
3. Find the network configuration that needs to be deleted. Support filtering search. Note the selection of network configuration type.
4. Click the Delete button. Just complete the deletion after secondary confirmation.
Note:
After deletion, this data engine will not be able to use this network configuration. If you need access, reconfiguration is required. Proceed with caution.
Modify Description Information
You can modify the description information of the configured network configuration by modifying the description information. Directions:
2. Enter Engine Management > Engine Network Configuration through the left sidebar.
3. Find the network configuration that needs to be deleted. Support filtering search. Note the selection of network configuration type.
4. Click the Modify Description Information button to edit.