tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Cloud Object Storage
    Documentation
    Download PDF

    Submitting Multi-File Zipping Job

    Last updated: 2024-06-12 15:27:04
    Download PDF

      Overview

      The multi-file zipping feature uses a POST request to zip multiple files into a compressed file such as a ZIP file. You can submit a job to zip multiple files and asynchronously return the compressed file.

      Billing Description

      Calling the API will incur multi-file zipping fees and COS read request fees as described in Request Fees.
      If the files are stored in COS STANDARD_IA storage class, calling the API will incur STANDARD_IA data retrieval fees as described in Data Retrieval Fees.
      Zipping is not supported for files stored in the ARCHIVE or DEEP ARCHIVE storage classes. To zip these files, you first need to restore them as instructed in POST Object restore.

      Restrictions

      Supported file quantity: At most 10,000 files can be zipped.
      Supported file size: The total size of the compressed file is less than 50 GB.
      Calling the API requires a signature. For more information, see Request Signature.

      Request

      Sample request

      POST /file_jobs HTTP/1.1
      Host: <BucketName-APPID>.ci.<Region>.myqcloud.com
      Date: <GMT Date>
      Authorization: <Auth String>
      Content-Length: <length>
      Content-Type: application/xml
      
      <body>
      Note:
      Authorization: Auth String (For more information, see Request Signature.)
      When this feature is used by a sub-account, relevant permissions must be granted as instructed in Authorization Granularity Details.

      Request headers

      This API only uses Common Request Headers.

      Request body

      This request requires the following request body:
      <Request>
          <Tag>FileCompress</Tag>
          <Operation>
              <FileCompressConfig>
                  <Prefix>test/</Prefix>
                  <Format>zip</Format>
                  <Flatten>0</Flatten>
              </FileCompressConfig>
              <Output>
                  <Region>ap-chongqing</Region>
                  <Bucket>test-1234567890</Bucket>
                  <Object>output/test.zip</Object>
              </Output>
              <UserData>This is my data.</UserData>
          </Operation>
          <QueueId>p2911917386e148639319e13c285cc774</QueueId>
          <CallBack>http://callback.demo.com</CallBack>
          <CallBackFormat>JSON<CallBackFormat>
      </Request>
      The nodes are described as follows:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Required
      Request
      None
      Request container
      Container
      Yes
      Request has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Required
      Tag
      Request
      Job type. It is FileCompress for multi-file zipping by default.
      String
      Yes
      Operation
      Request
      File zipping rule.
      Container
      Yes
      CallBackFormat
      Request
      Job callback format, which can be JSON or XML (default). It takes priority over that of the queue.
      String
      No
      CallBackType
      Request
      Job callback type, which can be Url (default) or TDMQ. It takes priority over that of the queue.
      String
      No
      CallBack
      Request
      Job callback address. It takes higher priority over that of the queue.
      String
      No
      CallBackMqConfig
      Request
      TDMQ configuration for job callback as described in Structure > CallBackMqConfig, which is required if CallBackType is TDMQ.
      Container
      No
      Operation has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Required
      FileCompressConfig
      Request.Operation
      File zipping rule
      Container
      Yes
      UserData
      Request.Operation
      The user information passed through, which is printable ASCII codes of up to 1,024 in length.
      String
      No
      Output
      Request.Operation
      Address where the compressed file is stored
      Container
      Yes
      FileCompressConfig has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Required
      Flatten
      Request.Operation.FileCompressConfig
      Whether to remove the directory structure of the original file during file zipping. Valid values:
      - 0: No. After file zipping, the original directory structure is retained.
      - 1: Yes. After file zipping, the original directory structure is removed and all files are at the same level.
      Assume that the original file URL is https://domain/source/test.mp4 and the original file directory is source/test.mp4. If this parameter is set to 1, the file directory in the ZIP file is test.mp4. If this parameter is set to 0, the file directory in the ZIP file is source/test.mp4.
      String
      Yes
      Format
      Request.Operation.FileCompressConfig
      Zipping type. Valid values: zip, tar, tar.gz
      String
      Yes
      UrlList
      Request.Operation.FileCompressConfig
      Files to be zipped can be sorted into an index file. The backend zips the files into a compressed file according to the file URL provided in the index file. The index file must be saved in the current bucket, and this field must provide the object address of the index file, for example, /test/index.csv.
      Index file format: Only CVS files in which one URL occupies one row (only files in the bucket are supported) are supported. If there are multiple columns, the first column is used as the URL by default. The number of files cannot exceed 10,000 and the total size cannot exceed 50 GB. Otherwise, the job will fail.
      String
      No
      Prefix
      Request.Operation.FileCompressConfig
      Files with a specified prefix in the bucket can be zipped. To zip files in a specified directory, add / to the directory. For example, to zip files in the test directory, set this parameter to test/. The number of files cannot exceed 10,000 and the total size cannot exceed 50 GB. Otherwise, the job will fail.
      String
      No
      Key
      Request.Operation.FileCompressConfig
      Multiple files in the bucket can be zipped. The number of files cannot exceed 1,000 and the total size cannot exceed 50 GB. Otherwise, the job will fail.
      String array
      No
      Note:
      Only one of UrlList, Prefix, and Key can be selected. They cannot all be empty or do not take effect at the same time. If multiple parameters are specified, the parameter with the highest priority prevails according to the priority UrlList > Prefix > Key.
      Output has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Required
      Region
      Request.Operation.Output
      Bucket region
      String
      Yes
      Bucket
      Request.Operation.Output
      Bucket where the compressed file is stored
      String
      Yes
      Object
      Request.Operation.Output
      Name of the compressed file
      String
      Yes

      Response

      Response headers

      This API only returns Common Response Headers.

      Response body

      The response body returns application/xml data. The following contains all the nodes:
      <Response>
          <JobsDetail>
              <Code>Success</Code>
              <Message/>
              <JobId>f93984788066911ed89ed352d4d9d2084</JobId>
              <State>Submitted</State>
              <CreationTime>2022-07-18T15:16:43+0800</CreationTime>
              <EndTime>-</EndTime>
              <StartTime>-</StartTime>
              <QueueId>p2911917386e148639319e13c285cc774</QueueId>
              <Tag>FileCompress</Tag>
              <Operation>
                  <FileCompressConfig>
                      <Prefix>test/</Prefix>
                      <Format>zip</Format>
                      <Flatten>0</Flatten>
                  </FileCompressConfig>
                  <Output>
                      <Region>ap-chongqing</Region>
                      <Bucket>test-1234567890</Bucket>
                      <Object>output/test.zip</Object>
                  </Output>
                  <UserData>This is my data.</UserData>
              </Operation>
          </JobsDetail>
      </Response>
      The nodes are described as follows:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Response
      None
      Result storage container
      Container
      Response has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      JobsDetail
      Response
      Job details
      Container
      JobsDetail has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Code
      Response.JobsDetail
      Error code, which is returned only if State is Failed
      String
      Message
      Response.JobsDetail
      Error message, which is returned only if State is Failed
      String
      JobId
      Response.JobsDetail
      Job ID
      String
      Tag
      Response.JobsDetail
      Job type. It is FileCompress for multi-file zipping by default.
      String
      State
      Response.JobsDetail
      Job status. Valid values: Submitted (submitted), Running (running), Success (successful), Failed (failed), Pause (paused), Cancel (canceled)
      String
      CreationTime
      Response.JobsDetail
      Job creation time
      String
      StartTime
      Response.JobsDetail
      Job start time
      String
      EndTime
      Response.JobsDetail
      Job end time
      String
      QueueId
      Response.JobsDetail
      ID of the queue where the job is in
      String
      Operation
      Response.JobsDetail
      Multi-file zipping rule
      Container
      Operation has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      UserData
      Response.JobsDetail.Operation
      The user information passed through.
      String
      Output
      Response.JobsDetail.Operation
      Same as Request.Operation.Output in the request.
      Container
      FileCompressConfig
      Response.JobsDetail.Operation
      Same as Request.Operation.FileCompressConfig in the request.
      Container
      FileCompressResult
      Response.JobsDetail.Operation
      Multi-file zipping result, which will not be returned when the job is not completed.
      Container
      FileCompressResult has the following sub-nodes:
      Node Name (Keyword)
      Parent Node
      Description
      Type
      Region
      Response.JobsDetail.Operation.FileCompressResult
      Region of the bucket where the compressed file is stored
      String
      Bucket
      Response.JobsDetail.Operation.FileCompressResult
      Bucket where the compressed file is stored
      String
      Object
      Response.JobsDetail.Operation.FileCompressResult
      Name of the compressed file
      String

      Error codes

      There are no specific error messages for this request operation. For common error messages, see Error Codes.

      Examples

      Request

      POST /file_jobs HTTP/1.1
      Authorization:q-sign-algorithm=sha1&q-ak=AKIDZfbOAo7cllgPvF9cXFrJD0a1ICvR****&q-sign-time=1497530202;1497610202&q-key-time=1497530202;1497610202&q-header-list=&q-url-param-list=&q-signature=28e9a4986df11bed0255e97ff90500557e0ea057
      Host:test-1234567890.ci.ap-chongqing.myqcloud.com
      Content-Length: 166
      Content-Type: application/xml
      
      <Request>
          <Tag>FileCompress</Tag>
          <Operation>
              <FileCompressConfig>
                  <Prefix>test/</Prefix>
                  <Format>zip</Format>
                  <Flatten>0</Flatten>
              </FileCompressConfig>
              <Output>
                  <Region>ap-chongqing</Region>
                  <Bucket>test-1234567890</Bucket>
                  <Object>output/test.zip</Object>
              </Output>
              <UserData>This is my data.</UserData>
          </Operation>
          <QueueId>p2911917386e148639319e13c285cc774</QueueId>
          <CallBack>http://callback.demo.com</CallBack>
          <CallBackFormat>JSON<CallBackFormat>
      </Request>

      Response

      HTTP/1.1 200 OK
      Content-Type: application/xml
      Content-Length: 230
      Connection: keep-alive
      Date: Mon, 18 Jul 2022 19:37:29 GMT
      Server: tencent-ci
      x-ci-request-id: NjMxMDJhYTNfMThhYTk0MGFfYmU1OV8zZjc=
      
      <Response>
          <JobsDetail>
              <Code>Success</Code>
              <Message/>
              <JobId>f93984788066911ed89ed352d4d9d2084</JobId>
              <State>Submitted</State>
              <CreationTime>2022-07-18T15:16:43+0800</CreationTime>
              <EndTime>-</EndTime>
              <StartTime>-</StartTime>
              <QueueId>p2911917386e148639319e13c285cc774</QueueId>
              <Tag>FileCompress</Tag>
              <Operation>
                  <FileCompressConfig>
                      <Prefix>test/</Prefix>
                      <Format>zip</Format>
                      <Flatten>0</Flatten>
                  </FileCompressConfig>
                  <Output>
                      <Region>ap-chongqing</Region>
                      <Bucket>test-1234567890</Bucket>
                      <Object>output/test.zip</Object>
                  </Output>
                  <UserData>This is my data.</UserData>
              </Operation>
          </JobsDetail>
      </Response>
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy