- Releases Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Practical Tutorial
- API Documentation
- Troubleshooting
- FAQs
- CFS Service Level Agreement
- Contact Us
- Glossary
- Releases Notes and Announcements
- Product Introduction
- Purchase Guide
- Getting Started
- Operation Guide
- Practical Tutorial
- API Documentation
- Troubleshooting
- FAQs
- CFS Service Level Agreement
- Contact Us
- Glossary
Overview
During your use of Cloud File Storage (CFS), you may need to copy data in many scenarios. This document provides recommended solutions for fast data copying in Linux operating systems.
Scenario 1: Data synchronization between CFS file systems, between different directories in a file system, and between a file system and a cloud disk
Data copying is required in many cases. The above three cases in scenario 1 are all about data copying between different directories from the perspective of a Linux operating system. Their operation methods are similar. You can use the
cp command for simple and fast copying. However, if there are numerous files, the copying will be very slow due to the single-threaded running mode of cp. In this case, we recommend you use the method provided in this document for copying to achieve concurrent acceleration.Scenario 2: Data synchronization between a file system and COS
When the public network is required for storage access or data import, it is recommended to use COS as a transit. You can use various basic tools provided by COS for data uploading and downloading. The COSBrowser and COSCMD tools are recommended.
Note
Prerequisites
The source and destination directories for copying exist on the CVM.
Note:
CFS supports cross-VPC access. You can connect multiple VPCs through Cloud Connect Network (CCN) and then mount and access a file system.
Directions
1. Download and install the Rclone tool.
wget https://downloads.rclone.org/v1.53.4/rclone-v1.53.4-linux-amd64.zip --no-check-certificateunzip rclone-v1.53.4-linux-amd64.zipchmod 0755 ./rclone-*/rclonecp ./rclone-*/rclone /usr/bin/rm -rf ./rclone-*
2. Run the following command to copy data:
rclone copy /mnt/src /mnt/dst -Pvv --transfers 32 --checkers 64 --copy-links --create-empty-src-dirs
This tool does not copy metadata information such as owner, ctime, and atime, by default. If you need to copy metadata information, use the
rsync command.Note:
The parameters in the command are described as follows (set
transfers and checkers as needed based on the system specifications):transfers: The number of concurrent file transfers (recommended up to twice the number of cores).
checkers: The number of concurrent scans of local files (recommended up to twice the number of cores).
P: Real-time display of the progress of data copying (the progress is refreshed every 500 ms. If
P is not added to the command, the progress is refreshed every minute).copy-links: Soft links for copying.
vv: Print copying logs.
create-empty-src-dirs: Copy empty directories.
3. After data copying is complete, you can view logs to check the results for different files.
Yes
No
Was this page helpful?