tencent cloud

Feedback

Access Logs

Last updated: 2023-12-29 14:46:39

    Overview

    Access logging is used to record access logs of domain names protected by WAF. It allows you to query and download access logs generated in the last 30 days and retain them for up to 180 days. After enabling this feature, you can query and download access logs as needed to meet your security compliance and OPS requirements.
    Note:
    To use access logging, you need to purchase an extra log services pack and enable access logging as instructed in Directions. Only after this feature is enabled for a domain name can its access requests be logged by WAF.
    To disable access logging: You can delete the corresponding billable item in Renewal Management. Note that access logging will be stopped within 2 hours and the history will be cleared within 24 hours once the billable item is removed.
    To expand logging capacity: When the required log storage exceeds the purchased log pack’s capacity, new access logs will not be stored, and the historical access logs will be deleted when the retention period reaches. To avoid loss of access logs from insufficient capacity, we recommend checking your log usage and expanding the capacity in advance.

    Directions

    Enabling access logging

    Log in to the WAF console and select Domain Name List on the left sidebar. Then toggle on the access logging switch for the domain name you choose.
    
    

    Viewing logs

    1. Log in to the WAF console and select Access Logs on the left sidebar. Then open the Log service tab.
    2. Click the drop-down list in the top left corner of the page to select domain names, and click OK.
    
    
    3. The usage capacity is displayed in top right corner. For more details about WAF billing, click Learn more.
    
    
    4. To view usage capacity and set the retention period at the same time, click Storage configuration, and then click Save to save your setting.
    Note:
    The retention period ranges from 1 to 30 days.
    

    Querying logs

    1. Log in to the WAF console and select Access Logs on the left sidebar. Then open the Log service tab.
    2. Search logs by using quick search, filters, or statements.
    Quick search: It allows you to search logs quickly be specifying a period.
    
    
    Search by filter: Select fields and operators, enter the filed values, and click OK. You can select multiple fields.
    
    
    Search by statement: It supports professional searches by statement and enables you to run more complex log queries. Enter the required information, and then click
    
    .
    
    
    Search statement
    Reserved Character
    Description
    AND
    "AND" logical operator, such as level:ERROR AND pid:1234
    OR
    "OR" logical operator, such as level:ERROR OR level:WARNING
    NOT
    "NOT" logical operator, such as level:ERROR NOT pid:1234
    TO
    "TO" logical operator, such as request_time:[0.1 TO 1.0]
    ""
    Double quotation mark, which quotes a phrase, such as name:"john Smith"
    Colon, which is used for key-value search, such as level:ERROR
    *
    Wildcard, which is used to replace zero, one, or more characters, such as host:www.test*.com
    ?
    Wildcard, which is used to replace one character, such as host:www.te?t.com
    ()
    Parentheses, which is used to group clauses to form sub queries and control the logic operations, such as (ERROR OR WARNING) AND pid:1234
    >
    Range operator, which indicates the left operand is greater than the right operand, such as status:>400
    >=
    Range operator, which indicates the left operand is greater than or equal to the right operand, such as status:>=400
    <
    Range operator, which indicates the left operand is less than the right operand, such as status:<400
    <=
    Range operator, which indicates the left operand is less than or equal to the right operand, such as status:<=400
    []
    Range operator, which includes the upper and lower boundary values, such as age:[20 TO 30]
    {}
    Range operator, which excludes the upper and lower boundary values, such as age:{20 TO 30}
    \\
    Escape character. An escaped character represents the literal meaning of the character, such as url:\\/images\\/favicon.ico. You can also use "" to wrap special characters as a whole, e.g., url:"/images/favicon.ico". For details about the difference between these two search methods, see Configuring Indexes.
    +
    Logical operator (similar to AND). The term +A indicates A must exist, such as +level:ERROR +pid:1234.
    -
    Logical operator (similar to NOT). The term -A indicates A does not exist, such as +level:ERROR -pid:1234.
    &&
    Logical operator (similar to AND), such as level:ERROR && pid:1234
    !
    Logical operator (similar to NOT), such as level:ERROR !pid:1234
    /
    Regular expression identifier in the format of /${regExp}/, e.g., /[mb]oat/ returns results containing moat or boat.
    _exists_
    _exists_:key returns results where the `key` value is not empty, e.g., _exists_:userAgent returns results where the userAgent value is not empty.
    ~
    Fuzzy search, e.g., level:errro~ returns results where level contains error.
    Note:
    The operators are case-sensitive. For example, AND and OR represent logical search operators, while and and or are regarded as common words.
    When multiple search statements are connected with spaces, they are regarded as in the OR logic. For example, warning error indicates to return results containing the warning keyword or error keyword.
    The following special characters must be escaped: +, -, &&, ||, !, ( ), { }, [ ], ^, ", ~, *, ?, :, \\
    Before performing a key:value search, make sure the key is configured in the index configuration of the log topic.
    Use () to group search conditions and clarify the precedency when using the "AND" and "OR" operators, such as (ERROR OR WARNING) AND pid:1234.

    Displaying logs

    1. Log in to the WAF console and select Access Logs on the left sidebar. Then open the Log service tab.
    2. Click Filed name to display the top five logs that match the filed.
    
    
    3. Click
    
    on the left of the date that the log is generated to view filed details. If you want to view details in JSON format, click JSON.
    
    
    JSON field description
    Field
    Description
    domain
    Wildcard domain name
    request_time
    Time that the client takes to send a request to WAF and receive a response
    uuid
    Unique identifier of an HTTP request
    schema
    Request protocol: HTTP or HTTPS
    method
    Client request method
    url
    Request URI, which resides between "/" and "?" in the client’s request path
    host
    Client domain name
    http_user_agent
    Request UA
    headers
    HTTP request header
    upstream_status
    Response code returned to WAF from the origin server
    status
    Response code returned to the client from WAF
    For CLB WAF, the response code 624 indicates the request is blocked and 600 indicates the request is allowed.
    
    For SAAS WAF, the response code 403 indicates the request is blocked and 200 indicates the request is allowed.
    body_bytes_sent
    Response body size
    upstream_response_time
    Time that WAF takes to receive the client request from the real server
    ip_info.country
    Country/Region
    ip_info.city
    City
    ip_info.province
    Province
    ip_info.operator
    ISP
    ip_info.ip_type
    IP type
    ip_info.idc
    IDC data center
    ip_info.longtitude
    Longitude
    ip_info.dimensionality
    Latitude
    
    4. Display the filtered log content in the list mode or field mode.
    Field mode: This is the default display mode. You can change to the other mode by clicking the icon in the top right corner.
    
    List mode: Click
    
    to change to list view.
    
    Field description
    Field
    Description
    msec
    Timestamp of when the request is sent
    schema
    Request protocol: HTTP or HTTPS
    method
    Client request method
    host
    Client domain name
    url
    Request URI, which resides between "/" and "?" in the client’s request path
    query
    HTTP Query String. The maximum length is 1 KB.
    body
    Request body data
    http_referer
    Page source
    http_user_agent
    Request UA
    http_x_forwarded_for
    All the proxies that pass the request
    cookie
    Request cookie. The maximum length is 1 KB.
    upstream_status
    Response code returned to WAF from the origin server
    upstream_response_time
    Time that WAF takes to receive the client request from the origin server
    upstream_addr
    Upstream server IP
    status
    Response code returned to the client from WAF
    upstream_status
    Response code returned to WAF from the origin server
    upstream_response_length
    Response length returned from the upstream server
    edition
    WAF versions: `sparta-waf`, `clb-waf`, `cdn-waf`

    Downloading access logs

    1. Log in to the WAF console and select Access Logs on the left sidebar. Then open the Log service tab.
    2. Click
    
    to enter the download page. Click OK to create a download task.
    Note:
    You cannot create more than one download task simultaneously.
    Up to 1 million logs can be downloaded at a time. To download more logs, it is recommended that you create multiple tasks to download them in batches.
    If you select a wildcard domain name (for example, *.abc.com), logs of all associated subdomain names such as those suffixed with .abc.com will also be downloaded.
    Up to five download tasks can be created.
    
    3. On the download page, click View Task to view the download details, such as the task number, creation time, and total number of logs.
    
    
    Log field description
    Field
    Description
    domain
    Wildcard domain name
    bytes_sent
    Response size, including response headers (in bytes) and downstream bandwidth
    method
    Client request method
    request_time
    Time that the client takes to send a request to WAF and receive a response
    http_connection
    HTTP request header Connection
    upstream_connect_time
    Time that WAF takes to send the client request to the real server
    uuid
    Unique identifier of an HTTP request
    upstream_addr
    Upstream server IP
    host
    Client domain name
    upstream_response_length
    Response length returned from the upstream server
    schema
    Request protocol: HTTP or HTTPS
    http_user_agent
    Request UA
    headers
    HTTP request header
    url
    Request URI, which resides between "/" and "?" in the client’s request path
    http_x_forwarded_for
    All the proxies that pass the request
    http_referer
    Page source
    body
    Request body data
    remote_addr
    Requester IP
    cookie
    Request cookie. The maximum length is 1 KB.
    bot_client_ip
    Client IP, which is typically the same as `remote_addr`
    request_length
    Request length
    http_accept
    HTTP request header Accept
    status
    Status code returned to the client from WAF
    protocol
    HTTP protocol, such as 1.1、1.0 and 2.0
    msec
    Timestamp of when the request is sent
    pipe
    Nginx built-in variable
    content_type
    HTTP request header Content-Type
    time_local
    Nginx readable local time string
    upstream_response_time
    Time that WAF takes to receive the client request from the real server
    server_addr
    WAF private IP
    edition
    WAF versions: `sparta-waf`, `clb-waf`, `cdn-waf`
    upstream_status
    Status code returned to WAF from the real server
    body_bytes_sent
    Response body size
    query
    HTTP Query String. The maximum length is 1 KB.
    
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support