For administrators only, the following operations may be involved.
1. Enter the DLC Console, click Go to CAM, and authorize Data Lake Computing.
2. In Role Management, click Grant.
Purchase Engine
Note:
You must have financial permissions in CAM.
1. On the DLC console, you can go to the engine purchase page from the Overview and Data Engine pages.
Overview > Initial Configuration > Purchase Data Engine:
Data Engine > Create Resource:
2. In the purchase page, select the Engine Type you want to buy:
Note:
SparkSQL: It is suitable for stable and efficient offline SQL tasks.
Spark job: It is suitable for native Spark stream/batch data job processing.
Presto: It is suitable for agile and fast interactive query and analysis.
Note:
A cluster size of 6CUs is relatively small, advisable only for testing scenarios. For actual production scenarios, it is recommended to select clusters with 64CUs or more.
Engine operation permissions are granted automatically
DLC supports default enablement of engine operation class permissions. Once enabled, all users will by default have the following permissions for that engine:
Utilize: Execute tasks using this engine.
Operation: Initiation of engine suspension or standby.
Monitoring: Administration of engine usage monitoring.
Note:
1. Upon termination, administrators inherently maintain all engine privileges. Ordinary users require an administrator to add permissions on the permission management page.
2. Existing ordinary user permissions will remain intact and can be deleted on the Permission Management page.
3. Subsequent newly created ordinary users have no usage rights, which should be manually added on the Permission Management page.
How do I enable or disable the self-delivery authorization engine
By default, the engine enables/disables two operation permission entries:
Access 2: Go to the Data Engine page and click Edit Auto-granting of engine permissions.
After setting engine permissions, click Confirm.
Team account activation
If you need multi-account collaborative product usage, you can activate it following these suggestions:
1. Permissions are not universal across different regions; separate permissions need to be set for each region.
2. Quick access to permission activation for Data Lake Compute:
To enable a sub-account's access to Data Lake Compute, please go to the CAM console for configuration.
To grant a sub-account read-write permissions for data and engines within Data Lake Compute, please go to the DLC console for setup.
Enable sub-account access to Data Lake Compute permissions
A primary account has all operation permissions for Data Lake Compute by default. The root account grants access permissions of Data Lake Compute to Sub-users through CAM, enabling Sub-users to have corresponding operation permissions for Data Lake Compute: QcloudDLCFullAccess(all operation permissions for Data Lake Compute).
2. Add a preset policy to the sub-account: QcloudDLCFullAccess(all operation permissions for Data Lake Compute).
You can search for the user to be authorized in the user list and click Authorize.
In the policy list, select QcloudDLCFullAccess(all operation permissions for Data Lake Compute).
Enable sub-account access to data and engine permissions in Data Lake Compute
Add a user to Data Lake Compute Permission Management
Note:
1. Please confirm the user permission effective region.
2. About User Segmentation:
Administrator: Has permissions for all resources.
Regular User: Needs specific permissions granted or must be assigned to a workgroup to obtain permissions.
1. log in to DLC Console, go to the Permission Management Page, select Corresponding Service Region, enter Permission Management Page. click Add User.
2. Add the account to DLC for management through the Sub-user's CAM ID. Please select the user type as needed.
3. Bind the user to the work group (optional step).
Note:
If you need to manage the permissions of multiple users, you can do so by binding them to a work group. After the work group is created, proceed with the addition. For the specific creation process, please refer to Quick Start with Permission Management in Data Lake Compute.
Add Engine and Data Permissions
After creating a user or work group, click 'Authorization Operation' in the list to add permissions to the work group, including data and engine permissions.
Data Permissions
Data Catalog Permissions: Includes the permissions to create databases and data directories under the data catalog.
Database Table Permissions: Grants fine-grained permissions at the database and table level, including the inquiry and editing of libraries, tables, views, functions, etc.
Engine permissions
Select engine permission policies according to the usage scenarios of users or work groups.
Note:
Utilize: Execute tasks using this engine.
Modification: Modify the engine's configuration parameters, such as specification configuration.
Operation: Initiation of engine suspension or standby.
Monitoring: Administration of engine usage monitoring.
Delete: Remove the engine.
Authorized: Once selected, all members under this Sub-user or workgroup will have the authorization permissions for the engine.
Configure Result Storage Location
Before using the Data Exploration feature, it is necessary to configure the Query Result Path. Once configured, the query results will be saved to the specified COS Path or DLC's Managed Storage. For detailed steps, please refer to the Configure Query Result Path Operation Guide.
Configure Result Storage Location
Enter the DLC Console, select the Data Exploration feature, choose Result Storage Location Configuration, and click Click to Configure.
Select the Storage Location and Method
Note:
Metadata Acceleration Bucket: In the current region, it can significantly improve query analysis performance. Inner tables can be enabled directly, while external tables require confirmation that engine permissions allow for it.
Please Note: Shared Engines cannot be associated with Metadata Acceleration Buckets. When a user selects a User Storage Path, a Dedicated Engine needs to bind to a Metadata Acceleration Bucket first before queries can take effect.
User Storage: User storage refers to your bucket path on COS.
Create Library
Before creating a database, choose the engine type to be used.
Enter the database name, and click Confirm.
Create a table
Native Table: These are tables stored in your DLC managed storage, by default in Iceberg format. Using native tables eliminates the need to deal with Iceberg's underlying files and offers capabilities such as data optimization to help build your data lake.
External Table: These tables represent data stored in your own COS buckets or other third-party data storage. DLC can directly create external tables for analysis without the need to load additional data.
Note:
After creating a Native Table, you need to refresh the browser to use it.
Data Query
Enter the DLC Console - Data Exploration. SQL Queries can be created on the analysis page. Features supported include Run All, Run Partially, Download Results, and materialized views.
Yes
No
Was this page helpful?