The volume file system is an important module of the Tencent HealthCare Omics Platform, specially designed for omics data analysis. The volume file system provides efficient and scalable shared CFS services, allowing multiple environments (instances) to share data through standard protocols for fast mounting. The volume file system supports high-speed processing and concurrent access of omics data, ensuring high availability and reliability when performing large-scale genetic data analysis. It is suitable for omics data processing scenes with strict performance and stability requirements.
In addition to being able to mount and use Volumes in computing tasks easily and quickly, the platform also provides a series of volume file management features. Among them, the creation, deletion, and one-click cleanup of Volumes are all completed in the console. For specific operation guide and Volume type selection guide, see Environment Management. Browsing Volume directories, transferring Volume files to buckets, and other management operations are completed in Omics Platform Website > File Management. This chapter will introduce them in detail.
Volume
The Volume (Volume) is the carrier of volume files and can be understood as the container for storing volume files. It is divided into the Standard type, High-Performance type, Standard Turbo, and High-Performance Turbo. Users can choose different types of Volumes as needed. Omics volume includes the following Storage Class. You can choose them according to the actual use cases:
Standard Type
Standard is a highly cost-effective file system that uses mixed media and accelerates data reads/writes through a data tiering mechanism. Three replicas on three independent physical servers on different racks are provided to guarantee strong consistency and successful storage of every data entry written to the file system. Its access server features hot data migration to ensure data reliability and high service availability, making it suitable for scenarios that require small-scale general data storage.
High-Performance Type
High-Performance is a low-latency file system that uses NVMe only and provides a high storage performance through a data tiering mechanism. Three replicas on three independent physical servers on different racks are provided to guarantee strong consistency and successful storage of every data entry written to the file system. Its access server features hot data migration to ensure data reliability and service high availability, making it suitable for small-scale core businesses that are latency-sensitive.
Standard Turbo Type
Standard Turbo is a parallel file system that uses mixed media and an asymmetric framework. Data nodes and metadata nodes are deployed independently. By allowing mounting with a private protocol, a single client can deliver performance like a storage cluster. In addition, underlying resources are isolated to ensure exclusive storage for the cluster. Three replicas on three independent physical servers on different racks are provided to guarantee strong consistency and successful storage of every data entry written to the file system. Its access server features hot data migration to ensure data reliability and service high availability, making it suitable for scenarios that require large-scale throughput and mixed loads.
High-Performance Turbo Type
High-Performance Turbo is a high-bandwidth, low-latency, parallel file system that uses NVMe only and an asymmetric framework. Data nodes and metadata nodes are deployed independently. By allowing mounting with a private protocol, a single client can deliver performance like a storage cluster. In addition, underlying resources are isolated to ensure exclusive storage for the cluster. Three replicas on three independent physical servers on different racks are provided to guarantee strong consistency and successful storage of every data entry written to the file system. Its access server features hot data migration to ensure data reliability and service high availability, making it suitable for scenarios that use a large number of small files.
Performance and Specifications
Storage Class
Standard
High-Performance
Standard Turbo
High-Performance Turbo
Product
Cost-effective, suitable for small-scale general data storage
High performance and low latency, suitable for small-scale latency-sensitive core businesses
High-throughput and large storage, suitable for businesses that require high throughput and mixed loads
High-throughput and high IOPS, suitable for businesses that use large-scale small files
Scenario
Small-scale enterprise file sharing, data backup/archive, and log storage
Small-scale CI/CD development and testing environments, high-performance web services, OLTP databases, and high-performance file sharing
Non-linear media asset editing, image rendering, AI inferencing, OLAP business, and high-performance computing
High-performance and large-scale computation, AI training, OLTP databases, big data analysis, and OLAP services
Storage Capacity
0 - 160TiB
0 - 32TiB
20TiB - 100PiB
10TiB - 100PiB
Bandwidth (MiB/s)
Min{100 + 0.1 x capacity (GiB), 300}
Min{200 + 0.2 x capacity (GiB), 1,024}
Min{0.1 x capacity (GiB), 100,000}
Min{0.2 x capacity (GiB), 100,000}
Read IOPS
min{2000+8*size(GiB),15000}
Min{2500 + 30 x capacity (GiB), 30,000}
Min{2 x capacity (GiB), 2 million}
Min{20 x capacity (GiB), 10 million}
Write IOPS
min{2000+8*size(GiB),15000}
Min{2500 + 30 x capacity (GiB), 30,000}
Min{1 x capacity (GiB), 1 million}
Min{5 x capacity (GiB), 3 million}
Maximum OPS
Read/write: 10,000/1,000
Read/write: 30,000/3,000
Read/Write: 300,000/20,000
Read/Write: 300,000/20,000
Latency
4K single-stream read: 3 ms
4K single-stream write: 7 ms
4K single-stream read: 1 ms
4K single-thread write: 1.5 ms
4K single-thread read: 0.2 ms
4K single-stream write: 3 ms
4K single-thread read: 0.2 ms
4K single-thread write: 1.5 ms
Supported Protocols
NFS/SMB
NFS
POSIX/MPI
POSIX/MPI
Scaling
Auto
Auto
Manual
Manual
Supported OS
Linux/Windows
Linux/Windows
Linux
Linux
Volume Management Permissions Description:
Only project operators or users with higher permissions have the permission to manage Volumes in the environment bound to their projects. If the manager needs to assign a Volume management permission to a specific user (group), the manager should bind this Volume environment in any project to which the user (group) belongs and grant the user (group) the above permissions as an operator. For operations related to binding the running environment, see Permission Management.
Browsing the Volume Directory
The Tencent HealthCare Omics Platform provides a Volume browser for browsing Volume file directories.
Enter File Management page and click Volume Files. The list on the left shows the volume list. The volumes in the managed environment support browsing the file directory through the volume browser. Currently, the volumes in the standard environment do not support browsing on the omics platform website. You can go to the Tencent Cloud CFS management page to view more details.
Select the Volume you want to view and click Start Volume Browser on the right side of the page to view the internal directory of the Volume. Click Monitoring above the Volume browser, and you can jump to the resource monitoring page to view the Volume usage details.
Note:
The Volume browser needs to start a computing instance (default specification: 1 core 1 GiB) to provide a running environment for performing file management tasks. After use, close the Volume browser on the page in time to avoid incurring additional fees. For detailed billing and price descriptions, see Product Pricing.
Sending Volume Files to a Bucket
You can send volume files to a bucket in two ways:
Method 1: Enter File Management page, click volume file, enter the Volume list page, select a Volume, and select the volume file to be sent in the Volume browser. Click the extension button Send to Bucket to start the corresponding data transfer job.
Method 2: Directly go to the Data Transfer page to create a new data transfer job. Choose the Volume to Bucket type to start the corresponding data transfer job.
There may be files with the same name under the target address. When starting the corresponding data transfer job, you need to specify how to handle files with the same name. Select Skip to retain the original file with the same name in the target address. Select Overwrite, and the file to be transferred will replace the original file in the target address.
Upload Local Files to the Volume
Enter File Management page, click the volume file, enter the Volume list page, select a Volume, start the Volume browser, and click Upload. Select the local file directory to be uploaded in the pop-up, and start uploading after confirmation. Batch uploading of local files is supported.
Other Operations
The volume browser supports a range of file management operations, including renaming, deleting, moving, creating new folders, and sending to buckets. All can be operated through extended options.
Yes
No
Was this page helpful?