tencent cloud

All product documents
Elastic MapReduce
DocumentationElastic MapReduceEMR Serverless HBase Operation GuideManaging an Instance Setting EC Policy
Setting EC Policy
Download PDF
Last updated: 2025-02-26 14:31:49
Setting EC Policy
Last updated: 2025-02-26 14:31:49
Download PDF
EC (Erasure Coding) policy is an advanced configuration of the instance. Enabling the EC policy can reduce the number of data replicas, thereby reducing resource costs.
Note：
The EC policy can be enabled or disabled as needed. It is disabled by default. Once it is enabled, new data is stored according to the RS-6-3-1024k policy.

Principle of EC

To ensure data reliability, HDFS's default data storage policy is 3 replicas, which means that when data is written, it occupies 3 times of the data size. This results in a significant waste of space. To address this issue, HDFS introduces the mature technology used in RAID disk arrays, EC (Erasure Coding).
EC is a data protection technology that splits the original data into multiple segments and adds a certain number of parity segments, allowing the original data to be restored even if some data is lost. Compared to traditional RAID technology, EC offers higher storage efficiency and lower redundancy.

RS-6-3-1024k Policy

Different EC encoding and decoding algorithms, block sizes, numbers of data blocks and check blocks can form different EC policies. The RS-6-3-1024k policy can save 50% of storage space compared to that of keeping 3 replicas, allowing up to 3 data blocks to be lost per data piece, and reducing network transmission by 50% during data writing and recovery.
Definition of RS-6-3-1024k Policy:
1. Using RS (Reed Solomon) encoding and decoding algorithms.
2. There are 6 DNs for storing data blocks.
3. There are 3 DNs for storing check blocks.
4. Loss of up to 3 blocks are allowed.
5. The size of each file block is 1024 KB (that is, 1 MB).
6. If the file stored using this EC policy is 100 MB, the total data volume written to the DataNode is (1+3/6) * 100 MB = 150 MB. Where:
The total size of data blocks is 100 MB.
The total size of check blocks is 3/6 * 100 MB = 50 MB.

Directions

1. Log in to the EMR Serverless HBase console, locate the instance to be terminated in the instance list, and click Instance ID to enter the Instance Information Page.
2. In Advanced Settings, find EC Policy, and click the switch to open the EC policy setting pop-up.
3. In the EC policy setting pop-up, click Confirm or Confirm Close for operation confirmation after confirming the risk of enabling or disabling the policy.
Note：
Only the EC policy RS-6-3-1024k is supported. The policy can be enabled only if the number of instance nodes is not less than 10.

Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback

Contact Us

Contact our sales team or business advisors to help your business.

Contact Us
Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

Submit a Ticket
7x24 Phone Support
Hong Kong, China
+852 800 906 020 (Toll Free)
United States
+1 844 606 0804 (Toll Free)
United Kingdom
+44 808 196 4551 (Toll Free)
Canada
+1 888 605 7930 (Toll Free)
Australia
+61 1300 986 386 (Toll Free)
EdgeOne hotline
+852 300 80699
More local hotlines coming soon