Before official and proper use of Data Lake Compute (DLC), you need to configure the initialization parameters and permissions in the target region in advance with the Tencent Cloud root account or DLC administrator account.
To avoid unnecessary errors, it is recommended that common users use DLC after the administrator completes the configurations.
Activating the DLC Service for the Root Account(Performed with the Tencent Cloud Root Account)
2. Click Go to CAM.
3. Authorize the activation of DLC Data Lake Service.
4. In Role Management, click Grant.
Creating a Tencent Cloud Sub-account and Adding DLC Service Policies Under the Root Account(Performed with the Tencent Cloud Root Account)
To enable multiple accounts to use the DLC service collaboratively, activate the DLC service as follows:
Creating a Tencent Cloud Sub-account
To enable a sub-account to access DLC, go to the Create User page of CAM for configuration. Adding the DLC Service Policy QcloudDLCFullAccess
2. Click Authorize.
3. In the pop-up window, enter DLC and select QcloudDLCFullAccess.
Adding the Sub-account as a DLC Account
Note:
Operation permission: The first time adding a sub-account as a DLC account, use the Tencent Cloud root account.
2. Select the user ID of the account you want to add, and specify the user type.
DLC Account Types and Permission Scopes
|
Data permissions | All permissions | No permission by default, and permissions need to be authorized by a DLC administrator. |
Standard engine | All permissions | No permission by default, and permissions need to be authorized by a DLC administrator. |
User management | Available | Unavailable |
Work group management | Available | Unavailable |
Authorization scope | All permissions | Authorizable permissions |
Causes of Addition Failures
If an error occurs when you add the account as a DLC account, check whether any of the following issues occur. If not, contact us for help. 1. Repeated addition: The account has already been added as a DLC account.
2. Account input error: An incorrect account is entered.
Adding a DLC Administrator Account
Note:
Operation permission: The first time adding a sub-account as a DLC administrator account, use the Tencent Cloud root account. For subsequent addition of sub-accounts as DLC administrator accounts, you can use the added DLC administrator account.
The add method is consistent with the previous chapter (adding an account as a DLC user). When selecting the type of user, choose DLC administrator.
DLC Administrator Account Types and Management Scope
|
Root account (owner) | CAM/DLC | Allowed | Allowed | Allowed | Allowed |
DLC administrator account | DLC | Disallowed | Disallowed | Allowed | Allowed |
Configuring the Storage Path of Query Results
Note:
Operation permission: Configured with the root account or DLC administrator account.
Enter Storage Configuration in the DLC console and configure the storage path for requry results on the Overview page or the Storage Configuration page. After the configuration is completed, the query results are stored in the specified Cloud Object Storage (COS) path or the managed storage device of DLC. Feature description:
1. DLC internal storage: The SELECT query results are stored in the DLC storage. The underlying storage is COS. The results can be stored for 36 hours.
2. User storage: The SELECT query results are stored in the bucket path on COS. You need to check whether the COS-related permissions are granted.
3. Metadata acceleration bucket: The performance of query and analysis in the local region can be improved.
4. For internal tables, the metadata acceleration bucket can be enabled directly. For external tables, you need to check whether the metadata acceleration bucket can be enabled based on the engine permission.
Note: A shared engine cannot be bound to a metadata acceleration bucket. When a user selects the user storage path, the exclusive engine needs to be bound to a metadata acceleration bucket before querying takes effect.
Purchasing an Engine
Note:
Operation permission: Purchased with the root account or an account having the financial permission.
You can purchase different types of engines based on your business requirements. Engines are classified into standard engines and SuperSQL engines. The two types of engines support different SQL syntaxes. The standard engine supports the native syntax and behavior, while the SuperSQL engine supports the DLC-developed SuperSQL syntax.
2. Click Create resource to enter the purchase page.
Note:
1. The engine is divided into the standard engine and SuperSQL engine. The difference between them is: They support different SQL syntaxes. The standard engine supports native syntax and behaviors, while the SuperSQL engine supports DLC's self-developed SuperSQL syntax.
2. Engine Specification Purchase Advice: Since a 16-CU cluster is relatively small in scale, it is advisable to use it only for testing scenarios. For real production scenarios, it is recommended to purchase a cluster with 64 CUs or more.
Standard Engine Permission Management
Note:
Operation permission: Configured with the root account or a sub-account having the CAM permissions.
Tencent Cloud CAM controls the DLC standard engine permissions. To ensure that the sub-account can use the DLC standard engine smoothly, you need to use the root account to authorize the sub-account the permissions. After the standard engine is created, all sub-accounts with the QcloudDLCFullAccess policy (all read/write and access permissions in the DLC console) have the permissions to use, manage, and monitor the standard engine. To achieve granular permission management of the standard engine, for example, user A only having the permission of engine A, you can create custom policies.
|
Scenario 1: The sub-account has all standard engine permissions. | Associate the preset QcloudDLCFullAccess policy with the sub-account. |
Scenario 2: The sub-account has partial standard engine permissions. | Create a custom policy. |
Granting All Standard Engine Permissions to a Sub-account
After you log in to the DLC console by using the root account or a sub-account having CAM operation permissions, find the sub-account in the sub-account list, click Authorize in the Operation column, search for and select QcloudDLCFullAccess, and click OK.
Granting Partial Standard Engine Permissions to a Sub-account
DLC supports resource-level authentication based on CAM tags. You can use tags to manage the existing standard engine resources and engine-related API permissions of DLC by category, achieving multidimensional resource management by category and granular authorization. For details about Tencent Cloud tags, see Tag. Based on Tencent Cloud tags, you can quickly achieve the following effects for DLC standard engine resources:
All users in department A can only use the standard engine resources associated with the tag for department A, and cannot use the standard engines associated with the tag for another department.
When users in department A create DLC standard engine resources, the resources should be associated with the tag for department A. If the resource is not associated with any tag or is associated with the tag for another department (rather than department A), the creation fails (optional).
Operation Steps
Step 1: Creating a Tag
1. Enter the Tag List page and click Create Tag. 2. Set Tag Key and Tag Value, click OK, and then a tag is created. For example, to create a tag with the department name Analyze, enter department for Tag Key and Analyze for Tag Value.
Step 2: Tagging the Standard Engine
Note:
Once a specific tag is associated with a standard engine, such as “department: Analyze” in the above example, only the users who are associated with that tag can use and manage the standard engine.
Step 3: Creating a Custom Policy
2. Click Create Custom Policy. In the pop-up window, select Authorize by Tag.
3. On the Visual Policy Generator tab, select DLC for Service and select All actions (dlc:*) for Action.
Note:
To restrict the permissions to terminate, create, or modify a standard engine, you can deselect the APIs under All actions. The APIs are described in the following table.
|
Unable to terminate an engine | DeleteDataEngine |
Unable to create an engine | CreateDataEngine |
Unable to modify an engine | UpdateDataEngine |
4. Select the “department: Analyze” tag created previously. By default, resource_tag under Select Condition Key is selected.
You can select request_tag,so that the users in the Analyze department will be required to associate the new DLC standard engine with the tag “department: Analyze” when creating an engine. For more introduction and usage of request_tag, see request_tag. 5. Click Next and CAM creates multiple split sub-policies. You can enter a name that is easy to search for a split sub-custom policy, such as “DLC-department-analyze-tag-policy”. Next to Authorized Users or Authorized User Groups, select the user/user group that needs to be associated with this custom policy. For example, associate all employee sub-accounts of the Analyze department in this example.
6. Click Complete and the custom policy is created. After the above custom policy is created, all DLC users associated with this custom policy can only access the standard engine tagged with “department: Analyze”.
Note:
1. To facilitate subsequent user maintenance, it is recommended that a user group be associated.
2. If a user associated with a custom policy has already been associated with the preset QcloudDLCFullAccess policy, the user still has all standard engine permissions.
SuperSQL Engine Permission Management
Note:
Operation permission: Configured by the root account or DLC admin.
Automatic Authorization Of Engine Operation Permissions (Only Supports SuperSQL Engine)
DLC supports enabling SuperSQL engine operation permissions by default. After enabling, all users will have the following permissions for this engine by default:
Usage: Use this engine for task execution.
Operation: Suspend or hang up the engine.
Monitoring: Monitor the usage and Ops of the engine.
Note:
1. After disabling, administrators will continue to have all engine permissions by default, while ordinary users require administrators to add permissions on the permission management page.
2. The original ordinary users' permissions are not affected and can be deleted by going toPermission Management page. 3. Subsequent newly created ordinary users will not have usage permissions and need to be manually added on the Permission Management page. How To Enable and Disable Automatic Authorization Engine Permissions
There are two permission entries for default enabling/disabling engine operation:
Entry 1:Engine Purchase Page >Advanced Configuration Item.
Entry 2: Go to the Engine Management page and click to edit authorization engine permissions. After setting the engine permissions, click Yes.
Activate Sub-Account Permissions For SuperSQL Engine In DLC
After creating a user or workgroup, click the authorization operation in the list to add permissions to the workgroup.
DLC Data Engines are divided into two categories: SuperSQL Engine and Standard Engine. For detailed differences and application scenarios, please refer to Data Engines. The earlier-launched SuperSQL Engine permissions are managed through the DLC console, where you can quickly manage SuperSQL Engine permissions in DLC Console > Permission Management. On the other hand, Standard Engine's permission management is uniformly controlled by Tencent Cloud CAM. For information on Standard Engine's permission management, please refer to DLC Permission Overview. For the SuperSQL Engine, based on the usage scenario of the user or workgroup, you can check the engine's permission policy in DLC Console > Permission Management > Engine Permissions.
Note:
Usage: Use this engine for task execution.
Modification: Modify the engine's configuration parameters, such as specification adjustment of the engine.
Operation: Suspend or hang up the engine.
Monitoring: Monitor the usage and Ops of the engine.
Deletion: Delete the engine.
Authorizable: After checking, all members under this Sub-user or workgroup have the authorization permission for the engine.
Data Permission Management
Note:
Operation permission: Configured with the root account or a DLC administrator account.
The root account or DLC administrator account can be used to configure permissions based on the usage scenario of users or work groups. On the Permission Management page, select the data permission policy. DLC data permissions include:
Data directory permissions
Permission for creating databases under the DataLakeCatalog directory and the permission for creating databases in other data directories.
Permission scope:
1. Whether to allow users or work groups to create databases under DataLakeCatalog
2. Whether to allow users or work groups to create databases in other data directories
Database and table permissions
Permissions for setting databases, data tables, views, and functions.
Permission scope:
|
Query and analysis permissions | Permissions to query all tables, views, and functions. create data tables in the database. | Query | Query |
Data editing permissions | Permissions to modify and delete databases and create tables. All permissions of all tables, views, and functions. | Data query, insertion, update, and deletion. Table modification and deletion. | Query, creation, modification, and deletion |
Owner permissions (permission is further granted based on the data editing permission) | Permissions to modify and delete databases and create tables. All permissions of all tables, views, and functions. | Data query, insertion, update, and deletion. Table modification and deletion. | Query, creation, modification, and deletion |
Advanced permission settings for databases and tables
If you select a single database, you can further set the query, insertion, update, and deletion permissions of tables, views, and functions under the database. If you select multiple databases, you can only set database permissions.
In advanced mode, you can set column-leve permissions. By selecting a single data table, you can add query permission for columns. You can also select one or more columns or all columns to grant permissions.
Row-level permissions
Based on database and table permissions, add row-level filter expressions to restrict the access scope.
DLC User Management
Note:
Operation permission: Configured with the root account or a DLC administrator account.
Modifying User Permissions
Select a user in the user list and click Edit to manage the user permissions. Viewing User Permissions
Click a user ID in the user list to enter the user details page. Deleting User Permissions
Remove specified permissions: In the user list on the Permission Management page, click Edit to enter the details page for modification. Delete users: In the user list on the Permission Management page, click Delete.
Work Group
Note:
Operation permission: Configured with the root account or a DLC administrator account.
To achieve better unified management and reduce administrators’ management costs, you can create a work group to add users in batch and authorize them in a unified manner.
Creating a Work Group
Adding Users to a Work Group
Method 1: When adding a user, select Bind to Working Group.
Method 2: On the Work group tab, click Edit to edit a work group.
Method 3: Add a user when creating a work group.
Adding Permissions to a Work Group
On the Work group tab, click Edit and add permissions.
Editing Work Group Members or Permissions
On the Work group tab, click Edit and manage the work group members or permissions.
Deleting a Work Group
On the Work group tab, click Delete for the work group.
FAQs
Why a failure occurs when I add the account as a DLC account?
If an error occurs when you add the account as a DLC account, first check whether any of the following issues occur. If not, please contact aftersales personnel for help.
1. Repeated addition: The account has already been added as a DLC account.
2. Account input error: An incorrect account is entered.
Why it displays “unavailable feature” or “no permission” when I enter the DLC console?
Your account may not have been added as a DLC account, or the administrator has not configured permissions for your account. You can contact your administrator to add your account as a DLC account and configure the relevant permissions for your account.
Which operations must be performed with the root account during the use of DLC?
The root account is responsible for activating the DLC service, creating Tencent Cloud sub-accounts, authorizing the policy QcloudDLCFullAccess for sub-accounts, designating the first DLC administrator, and granting financial permissions.
How are the permissions determined if the permissions of a user are inconsistent with those of the work group to which the user belongs?
The permissions are the union of user and work group permissions.
