Managed Storage Configuration

Recent Pages

Managed Storage Configuration

Last updated: 2024-07-31 17:30:11

Managed storage refers to the storage space hosted on the Data Lake product, with COS as the underlying storage. Managed storage contains data such as native tables, user program packages, and query results. Therefore, to utilize the capabilities of native tables and data optimization, it is necessary to enable managed storage first. The native tables on managed storage are by default in the Iceberg format, so you don’t need to manage the underlying file contents. For details on managed storage billing, please refer to Billing Overview.
This document introduces how to enable and configure managed storage.
Enable Managed Storage
Step 1: Enter Managed Storage Configuration
You can enter the managed storage configuration in the Data Exploration module or the Global Configuration > Storage Configuration module.
﻿
﻿
﻿
Step 2: Open Managed Storage
1. Check to enable managed storage and save.
Here, you can specify the managed storage type as either a Metadata Acceleration Bucket or an Ordinary Bucket. The billing for both is consistent, but it is necessary to separately configure engine access permissions for the Metadata Acceleration Bucket. For details, please refer to Binding of Metadata Acceleration Bucket.
2. The query result path is used to temporarily store SQL query results, Spark Job Shuffle data, etc. You need to specify a path to ensure the normal operation of jobs and tasks. If you have enabled managed storage, it is recommended to configure the query result path as Managed Storage. You can also configure the query result path to your own account's COS bucket path.
﻿
﻿
﻿
View managed bucket
After enabling managed storage, a bucket will be created, and you can view the buckets and data on managed storage in the Data Management module.
﻿
﻿
﻿
Destroy Managed Storage
Destroying data is a high-risk action; only after all database table data has been deleted, can you proceed to destroy managed storage. Destroying managed storage requires administrator privileges.
Step one: Delete database table data
To destroy managed storage, you must first delete all database table data on the managed storage.
You can refer to the Data Catalog and DMC and Data Table Management documents to delete the database table data, or you can run the DROP Syntax in the Data Exploration module to delete the database table data.
Step two: Destroy Managed Storage
After deleting the database table data, you can destroy managed storage on the managed storage configuration tab under the Storage Configuration module.
Destroying managed storage will delete all DLC managed buckets, so please proceed with caution.
﻿