Push to AWS S3 Compatible COS

Last updated: 2024-06-28 14:21:49

EdgeOne Real-time Log Push can be configured through the console or API to push logs to AWS S3 Signature Version 4 Authentication Algorithm compatible COS, such as:
﻿Tencent Cloud COS﻿
﻿AWS S3﻿
﻿Google Cloud Storage﻿
﻿IBM Cloud Object Storage﻿
﻿Linode Object Storage﻿
﻿Oracle Cloud Object Storage, etc
Operation step
1. Access the EdgeOne console, locate the left-hand menu, and select the Site List. Within this listing, click on the Site requiring configuration to enter its detailed page.
2. On the Site Details Page, click Log Service > Real-time Logs.
3. On the Real-time Logs page, click Create shipping task.
4. On Select the log source Page, fill in the Task name, select the Log type Service area, the Domain name/Layer 4 Proxy Instance for which the logs need to be pushed, and click Next.
Note:
Currently, only Site Acceleration Logs and Four-layer Proxy Logs are supported to be pushed to S3 Compatible COS.
5. On the Define delivery content page:
(Required) Check the log fields to be pushed from the Predefined field list;
(Optional) Add a Custom Log Field, supporting the extraction of specified field names from the Request Header, Response Header, and Cookie Header;
(Optional) Configure the Push Log Filter Criteria, the default is to push full logs;
(Optional) In advanced configuration, set the sampling ratio. By default, sampling is not enabled, and 100% of logs are pushed to the destination.
(Optional) In advanced configuration, set the log output format. The default format is JSON Lines.
6. On the Select a destination address page, select  S3 compatible , click  Next .
7. On the Destination Information Page, enter the related destination and parameter information.
Parameter name
Description
Endpoint URL
URL without bucket name or path, e.g.: https://cos.ap-nanjing.myqcloud.com.
bucket region	
Region where the bucket is located, for example:ap-nanjing.
Bucket
Bucket name and log storage directory, for example:your_bucket_name/EO-logs/. Whether or not you end the directory with /, it will be correctly parsed and processed.
File compression	
After checking, gzip will be used to compress log files.
Secret ID	
Access Key ID used to access the bucket.
Secret key	
Secret key used to access the bucket.
8. Click Ship, confirm the related cost alert in the popup, and click Confirm Creation.
9. After issuing the real-time log push task, EdgeOne will push a test file to the target bucket directory to verify connectivity, for example, 1699874755_edgeone_push_test.txt, with the file content being the fixed string "test".
File Name Description
Logs will be stored in the specified bucket directory in the format {{UploadTime}}_{{Random}}.log, and will be archived by date (UTC+00:00) into folders. For example: 20230331/20230331T185917Z_2aadf5ce.log. When gzip compression is enabled, the file name will be 20230331/20230331T185917Z_2aadf5ce.log.gz.
UploadTime: The upload time of the log file, using ISO-8601 Format, UTC+00:00 Time Zone.
Random: Random characters used to identify different files when there are multiple log files for the same upload time due to large log volume.
﻿

tencent cloud

Recent Pages

Push to AWS S3 Compatible COS

Operation step

File Name Description

Was this page helpful?

Was this page helpful?

Parameter name	Description
Endpoint URL	URL without bucket name or path, e.g.: `https://cos.ap-nanjing.myqcloud.com`.
bucket region	Region where the bucket is located, for example:`ap-nanjing`.
Bucket	Bucket name and log storage directory, for example:`your_bucket_name/EO-logs/`. Whether or not you end the directory with `/`, it will be correctly parsed and processed.
File compression	After checking, gzip will be used to compress log files.
Secret ID	Access Key ID used to access the bucket.
Secret key	Secret key used to access the bucket.

tencent cloud

Sign Up

Log in

Recent Pages

Push to AWS S3 Compatible COS

Operation step

File Name Description

Was this page helpful?

Was this page helpful?