API | Operation | Description |
Downloading an object | Downloads an object to local |
download_file(Bucket, Key, DestFilePath, PartSize=20, MAXThread=5, EnableCRC=False, **Kwargs)
# -*- coding=utf-8from qcloud_cos import CosConfigfrom qcloud_cos import CosS3Clientfrom qcloud_cos.cos_exception import CosClientError, CosServiceErrorimport sysimport osimport logging# In most cases, set the log level to INFO. If you need to debug, you can set it to DEBUG and the SDK will print information about the communication with the server.logging.basicConfig(level=logging.INFO, stream=sys.stdout)# 1. Set user attributes such as secret_id, secret_key, and region. Appid has been removed from CosConfig and thus needs to be specified in Bucket, which is formatted as BucketName-Appid.secret_id = os.environ['COS_SECRET_ID'] # User `SecretId`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.secret_key = os.environ['COS_SECRET_KEY'] # User `SecretKey`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.region = 'ap-beijing' # Replace it with the actual region, which can be viewed in the console at https://console.tencentcloud.com/cos5/bucket.# For the list of regions supported by COS, visit https://www.tencentcloud.com/document/product/436/6224.token = None # Token is required for temporary keys but not permanent keys. For more information about how to generate and use a temporary key, visit https://www.tencentcloud.com/document/product/436/14048.scheme = 'https' # Specify whether to use HTTP or HTTPS protocol to access COS. This field is optional and is `https` by default.config = CosConfig(Region=region, SecretId=secret_id, SecretKey=secret_key, Token=token, Scheme=scheme)client = CosS3Client(config)# Use the advanced API to download once without retry. In this case, the checkpoint restart feature is not used.response = client.download_file(Bucket='examplebucket-1250000000',Key='exampleobject',DestFilePath='local.txt')# Use the advanced API for checkpoint restart, and the successfully uploaded parts will not be downloaded again during retries (10 retries here).for i in range(0, 10):try:response = client.download_file(Bucket='examplebucket-1250000000',Key='exampleobject',DestFilePath='local.txt')breakexcept CosClientError or CosServiceError as e:print(e)
def upload_percentage(consumed_bytes, total_bytes):"""Callback function that calculates the upload progress (in percentage):param consumed_bytes: uploaded data amount:param total_bytes: total data amount"""if total_bytes:rate = int(100 * (float(consumed_bytes) / float(total_bytes)))print('\\r{0}% '.format(rate))sys.stdout.flush()response = client.download_file(Bucket='examplebucket-1250000000',Key='exampleobject',DestFilePath='local.txt',PartSize=1,MAXThread=5,EnableCRC=True,TrafficLimit='1048576'IfMatch='"9a4802d5c99dafe1c04da0a8e7e166bf"',IfModifiedSince='Wed, 28 Oct 2014 20:30:00 GMT',IfNoneMatch='"9a4802d5c99dafe1c04da0a8e7e166bf"',IfUnmodifiedSince='Wed, 28 Oct 2014 20:30:00 GMT',ResponseCacheControl='string',ResponseContentDisposition='string',ResponseContentEncoding='string',ResponseContentLanguage='string',ResponseContentType='string',ResponseExpires='string',VersionId='string',progress_callback=upload_percentage)
Parameter | Description | Type | Required |
Bucket | Bucket name in the format of BucketName-APPID | String | Yes |
Key | Object key, which uniquely identifies an object in a bucket. For example, if an object’s access endpoint is examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/doc/pic.jpg , its key is doc/pic.jpg . | String | Yes |
DestFilePath | Local path to save the downloaded file | String | Yes |
PartSize | Part size for multipart download. Default value: 20 MB | Int | No |
MAXThread | Maximum number of concurrent threads for a multipart download. Default value: 5 | Int | No |
EnableCRC | Whether to enable CRC for local and remote files. Default: False | Bool | No |
TrafficLimit | Bandwidth limit in bit/s for a single request (or a single thread for the advanced download API). Value range: 819200 - 838860800, i.e., 100 KB/s - 100 MB/s | String | No |
IfMatch | ETag that needs to be matched. The object is downloaded only if its ETag matches the value. | String | No |
IfModifiedSince | Required modification time. The object is downloaded only if it has been modified after the specified time. | String | No |
IfNoneMatch | ETag that cannot be matched. The object is downloaded only if its ETag does not match the value. | String | No |
IfUnmodifiedSince | Required unmodified time. The object is downloaded only if it has not been modified since the specified time. | String | No |
ResponseCacheControl | Cache-Control of the response header | String | No |
ResponseContentDisposition | Content-Disposition of the response header | String | No |
ResponseContentEncoding | Content-Encoding of the response header | String | No |
ResponseContentLanguage | Content-Language of the response header | String | No |
ResponseContentType | Content-Type of the response header | String | No |
ResponseExpires | Expires of the response header | String | No |
VersionId | Version ID of the object to download | String | No |
progress_callback | Callback function for the upload progress. You can customize this function to query the upload progress. | Func | No |
# -*- coding=utf-8import osimport loggingimport sysimport jsonimport osfrom qcloud_cos import CosConfig, CosServiceErrorfrom qcloud_cos import CosS3Clientfrom qcloud_cos.cos_threadpool import SimpleThreadPool# In most cases, set the log level to INFO. If you need to debug, you can set it to DEBUG and the SDK will print information about the communication with the server.logging.basicConfig(level=logging.INFO, stream=sys.stdout)# 1. Set user attributes such as secret_id, secret_key, and region. Appid has been removed from CosConfig and thus needs to be specified in Bucket, which is formatted as BucketName-Appid.secret_id = os.environ['COS_SECRET_ID'] # User `SecretId`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.secret_key = os.environ['COS_SECRET_KEY'] # User `SecretKey`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.region = 'ap-beijing' # Replace it with the actual region, which can be viewed in the console at https://console.tencentcloud.com/cos5/bucket.# For the list of regions supported by COS, visit https://www.tencentcloud.com/document/product/436/6224.token = None # Token is required for temporary keys but not permanent keys. For more information about how to generate and use a temporary key, visit https://www.tencentcloud.com/document/product/436/14048.scheme = 'https' # Specify whether to use HTTP or HTTPS protocol to access COS. This field is optional and is `https` by default.config = CosConfig(Region=region, SecretId=secret_id, SecretKey=secret_key, Token=token, Scheme=scheme) # Obtain the object to configureclient = CosS3Client(config)# Bucket informationtest_bucket = 'examplebucket-1250000000'start_prefix = 'data/'# COS uses slashes (/) to indicate folders.# The delimiter parameter is left empty by default, which lists all subdirectories under the directories to achieve the recursive effect of a local folder.# If the delimiter parameter is set to "/", you need to handle subdirectories recursively in the SDK.delimiter = ''# List and return information about all subdirectories under the current directory.def listCurrentDir(prefix):file_infos = []sub_dirs = []marker = ""count = 1while True:response = client.list_objects(test_bucket, prefix, delimiter, marker)# Debug the output.# json_object = json.dumps(response, indent=4)# print(count, " =======================================")# print(json_object)count += 1if "CommonPrefixes" in response:common_prefixes = response.get("CommonPrefixes")sub_dirs.extend(common_prefixes)if "Contents" in response:contents = response.get("Contents")file_infos.extend(contents)if "NextMarker" in response.keys():marker = response["NextMarker"]else:breakprint("=======================================================")# If the delimiter parameter is set to "/", you need to handle subdirectories recursively.# sorted(sub_dirs, key=lambda sub_dir: sub_dir["Prefix"])# for sub_dir in sub_dirs:# print(sub_dir)# sub_dir_files = listCurrentDir(sub_dir["Prefix"])# file_infos.extend(sub_dir_files)print("=======================================================")sorted(file_infos, key=lambda file_info: file_info["Key"])for file in file_infos:print(file)return file_infos# Download files to a local directory. Local files with the same name (if any) will be overwritten.# If the directory structure does not exist, a directory structure will be created according to that in COS.def downLoadFiles(file_infos):localDir = "./download/"pool = SimpleThreadPool()for file in file_infos:# Download files to local.file_cos_key = file["Key"]localName = localDir + file_cos_key# If the local directory does not exist, create it recursively.if not os.path.exists(os.path.dirname(localName)):os.makedirs(os.path.dirname(localName))# skip dir, no need to download itif str(localName).endswith("/"):continue# Download files# Use a thread pool.pool.add_task(client.download_file, test_bucket, file_cos_key, localName)# Download in whole# response = client.get_object(# Bucket=test_bucket,# Key=file_cos_key,# )# response['Body'].get_stream_to_file(localName)pool.wait_completion()return None# Encapsulate the feature. Download a COS directory to local.def downLoadDirFromCos(prefix):global file_infostry:file_infos = listCurrentDir(prefix)except CosServiceError as e:print(e.get_origin_msg())print(e.get_digest_msg())print(e.get_status_code())print(e.get_error_code())print(e.get_error_msg())print(e.get_resource_location())print(e.get_trace_id())print(e.get_request_id())downLoadFiles(file_infos)return Noneif __name__ == "__main__":downLoadDirFromCos(start_prefix)
GET Object
) is used to download an object.get_object(Bucket, Key, **kwargs)
# -*- coding=utf-8from qcloud_cos import CosConfigfrom qcloud_cos import CosS3Clientimport sysimport osimport logging# In most cases, set the log level to INFO. If you need to debug, you can set it to DEBUG and the SDK will print information about the communication with the server.logging.basicConfig(level=logging.INFO, stream=sys.stdout)# 1. Set user attributes such as secret_id, secret_key, and region. Appid has been removed from CosConfig and thus needs to be specified in Bucket, which is formatted as BucketName-Appid.secret_id = os.environ['COS_SECRET_ID'] # User `SecretId`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.secret_key = os.environ['COS_SECRET_KEY'] # User `SecretKey`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.region = 'ap-beijing' # Replace it with the actual region, which can be viewed in the console at https://console.tencentcloud.com/cos5/bucket.# For the list of regions supported by COS, visit https://www.tencentcloud.com/document/product/436/6224.token = None # Token is required for temporary keys but not permanent keys. For more information about how to generate and use a temporary key, visit https://www.tencentcloud.com/document/product/436/14048.scheme = 'https' # Specify whether to use HTTP or HTTPS protocol to access COS. This field is optional and is `https` by default.config = CosConfig(Region=region, SecretId=secret_id, SecretKey=secret_key, Token=token, Scheme=scheme)client = CosS3Client(config)response = client.get_object(Bucket='examplebucket-1250000000',Key='exampleobject')response['Body'].get_stream_to_file('exampleobject')
# -*- coding=utf-8from qcloud_cos import CosConfigfrom qcloud_cos import CosS3Clientimport sysimport osimport logging# In most cases, set the log level to INFO. If you need to debug, you can set it to DEBUG and the SDK will print information about the communication with the server.logging.basicConfig(level=logging.INFO, stream=sys.stdout)# 1. Set user attributes such as secret_id, secret_key, and region. Appid has been removed from CosConfig and thus needs to be specified in Bucket, which is formatted as BucketName-Appid.secret_id = os.environ['COS_SECRET_ID'] # User `SecretId`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.secret_key = os.environ['COS_SECRET_KEY'] # User `SecretKey`. We recommend you use a sub-account key and follow the principle of least privilege to reduce risks. For information about how to obtain a sub-account key, visit https://www.tencentcloud.com/document/product/598/32675.region = 'ap-beijing' # Replace it with the actual region, which can be viewed in the console at https://console.tencentcloud.com/cos5/bucket.# For the list of regions supported by COS, see https://www.tencentcloud.com/document/product/436/6224token = None # Token is required for temporary keys but not permanent keys. For more information about how to generate and use a temporary key, visit https://www.tencentcloud.com/document/product/436/14048.scheme = 'https' # Specify whether to use HTTP or HTTPS protocol to access COS. This field is optional and is `https` by default.config = CosConfig(Region=region, SecretId=secret_id, SecretKey=secret_key, Token=token, Scheme=scheme)client = CosS3Client(config)response = client.get_object(Bucket='examplebucket-1250000000',Key='exampleobject',Range='bytes=0-100')response['Body'].get_stream_to_file('exampleobject')
response = client.get_object(Bucket='examplebucket-1250000000',Key='exampleobject',Range='string',IfMatch='"9a4802d5c99dafe1c04da0a8e7e166bf"',IfModifiedSince='Wed, 28 Oct 2014 20:30:00 GMT',IfNoneMatch='"9a4802d5c99dafe1c04da0a8e7e166bf"',IfUnmodifiedSince='Wed, 28 Oct 2014 20:30:00 GMT',ResponseCacheControl='string',ResponseContentDisposition='string',ResponseContentEncoding='string',ResponseContentLanguage='string',ResponseContentType='string',ResponseExpires='string',VersionId='string',TrafficLimit='819200')
Parameter | Description | Type | Required |
Bucket | Bucket name in the format of BucketName-APPID | String | Yes |
Key | Object key, which uniquely identifies an object in a bucket. For example, if an object’s access endpoint is examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/doc/pic.jpg , its key is doc/pic.jpg . | String | Yes |
Range | Byte range of the object to download. Format: bytes=first-last | String | No |
IfMatch | ETag that needs to be matched. The object is downloaded only if its ETag matches the value. | String | No |
IfModifiedSince | Required modification time. The object is downloaded only if it has been modified after the specified time. | String | No |
IfNoneMatch | ETag that cannot be matched. The object is downloaded only if its ETag does not match the value. | String | No |
IfUnmodifiedSince | Required unmodified time. The object is downloaded only if it has not been modified since the specified time. | String | No |
ResponseCacheControl | Cache-Control of the response header | String | No |
ResponseContentDisposition | Content-Disposition of the response header | String | No |
ResponseContentEncoding | Content-Encoding of the response header | String | No |
ResponseContentLanguage | Content-Language of the response header | String | No |
ResponseContentType | Content-Type of the response header | String | No |
ResponseExpires | Expires of the response header | String | No |
VersionId | Version ID of the object to download | String | No |
TrafficLimit | Bandwidth limit in bit/s for a single request (or a single thread for the advanced download API). Value range: 819200 - 838860800, i.e., 100 KB/s - 100 MB/s | String | No |
{'Body': StreamBody(),'ETag': '"9a4802d5c99dafe1c04da0a8e7e166bf"','Last-Modified': 'Wed, 28 Oct 2014 20:30:00 GMT','Accept-Ranges': 'bytes','Content-Range': 'bytes 0-16086/16087','Cache-Control': 'max-age=1000000','Content-Type': 'application/octet-stream','Content-Disposition': 'attachment; filename="filename.jpg"','Content-Encoding': 'gzip','Content-Language': 'zh-cn','Content-Length': '16807','Expires': 'Wed, 28 Oct 2019 20:30:00 GMT','x-cos-meta-test': 'test','x-cos-version-id': 'MTg0NDUxODMzMTMwMDM2Njc1ODA','x-cos-request-id': 'NTg3NzQ3ZmVfYmRjMzVfMzE5N182NzczMQ=='}
Parameter | Description | Type |
Body | Content of the downloaded object. You can use the `get_raw_stream()` method to get a file stream and the `get_stream_to_file(local_file_path)` method to download the object’s content to a specified local file. | StreamBody |
ETag | MD5 checksum of the object | String |
Last-Modified | Time when the object was last modified | String |
Accept-Ranges | Range unit. HTTP standard header | String |
Content-Range | Content range. HTTP standard header | String |
Cache-Control | Cache policy. HTTP standard header | String |
Content-Type | Content type. HTTP standard header | String |
Content-Disposition | Filename. HTTP standard header | String |
Content-Encoding | Encoding format. HTTP standard header | String |
Content-Language | Language type. HTTP standard header | String |
Content-Length | Object size | String |
Expires | Cache expiration time. HTTP standard header | String |
x-cos-meta-* | User-defined object metadata. It must start with `x-cos-meta`; otherwise, it will be ignored. | String |
x-cos-version-id | Version ID of the object if versioning is enabled | String |
Was this page helpful?