Access logging
What is access logging?
Access logging records details of requests made to object storage buckets. Access logs can be used for security and access auditing purposes, as well as analyzing client requests to your bucket.
For details on how to set up access logging, please refer to Configure access logging.
Each record in the access log represents one request, each field is separated by blank. If there is no data or is unknown, it is expressed as -.
Each field is described in detail in Log record field.
When a request comes to the bucket, logs are recorded within a few hours and the logs are stored in the target bucket.
When the bucket logging status changes, logs are stored in stages over a certain period of time, and perfect transmission may not be guaranteed depending on the storage environment.
Bucket role permissions for log transfer
To transfer logs from the source bucket to the destination bucket, the credentials of the Object Storage Agent service account will be used to store the logs.
The Object Storage Agent service account in the target bucket must have storage.buckets.get and storage.buckets.update permissions of the Object Storage role to store logs properly.
The Object Storage Agent account is a service account and has the Project Member role(IAM).
When you create a bucket, by default, the IAM Project Member group has Storage Editor permission, so log transmission is possible without separate permission settings.
If the project member group is deleted from the permissions in the role of the target bucket, the Object Storage Agent account belonging to the project member group loses Storage Editor permission and cannot save logs.
In this case, you must manually add the Object Storage Agent service account with a role of Storage Editor or higher as shown below.
Object key format of log
Object keys in access log can generate logs using the following format:
- SimplePrefix:
DestinationPrefixYYYY-MM-DD-hh-mm-ss-UniqueString - PartitionedDateSource:
DestinationPrefixSourceAccountId/SourceRegion/SourceBucket/YYYY/MM/DD/YYYY-MM-DD-hh-mm-ss-UniqueString
A detailed description of the object key format is as follows.
YYYY,MM,DD,hh,mm,ss: year, month, day, hour, minute, second (Coordinated Universal Time (UTC))DestinationPrefix: Destination prefixProjectID: Project IDSourceRegion: Region nameSourceBucketsource bucketUniqueString: Object key identification string
UniqueString is a separator value to prevent logs from being overwritten.
Log record fields
The following is a description of the log field records recorded in the access log.
1.Domain ID (domain_id)
The customer's domain ID value.
예) 327373ec52974577a79a5e26b26c27e9
2. Project ID (project_id)
This is the project ID value under the customer's domain.
Example) ca7f6c731a004091a32d4eb97ec17271
3. Bucket name (bucket)
The name of the bucket served by the request.
Example) Kakao-bucket
4.Bucket owner ID (bucket_owner)
Owner ID of the origin bucket.
Example) 54ba02ba408d4968a35686e48db85ea8
5.Time
This is the request time to the bucket. Use the [%d/%B/%Y:%H:%M:%S %z] format. %d: two-digit day %b: Abbreviated month name (e.g. May) %Y: four-digit year %H: Two digit hour %M: Two-digit minute %S: Two-digit second %z: UTC offset This date and time is in Coordinated Universal Time (UTC).
Example) 16/May/2024:08:20:05 +0000
6.Remote IP (remote_ip)
The client IP that made the request to the bucket.
Example) 127.0.0.1
7.Requester ID (user_id)
This is the requester ID that requested the bucket. When accessing from the public without a separate authentication token, it is displayed as [-].
Example) 0e26ca49d2ca4bbfbd85e5901545c796
8.Request ID (request_id)
ID generated to identify the request.
Example) tx000008b923132a7716acd-0065795106-8fb2f-kr-central-2
9.Operation
The operations listed here are declared as [REST.{HTTP_method}.{resource_type}]. Objects deleted by Lifecycle policy are not logged.
Example) REST.POST.OBJECT
10.Object key (key)
The requested object key.
Example) /Image/kakaocloud/ryan.jpg
11.Request-URI (request_uri)
Request-URI of the HTTP request message.
Example) /v1/1b5e24ba80104e9f9aecd2bcfeb7da2/object-reg-test-1/mulit-object?uploads
12.HTTP status (http_status)
The HTTP status code of the response.
ex) 200
13.Error code (error_code)
This is the error code of object storage. If there is no error, it is displayed as [-]. (Logged only for S3 API calls.)
yes) -
14.Request bytes (request_body_size)
Number of requested bytes received.
Example) 2662992
15.Response bytes (response_body_size)
Number of response bytes sent excluding HTTP protocol overhead.
Example) 5432290
16.Object size (object_size)
The overall size of the object.
Example) 7452918
17.Total time (total_time)
Time spent by Object Storage on user requests. Expressed in milliseconds. It is measured from the time a byte of the request is received to the time the last byte of the response is transmitted.
Example) 253.507608ms
18.HTTP Referer (http_referer)
This is the HTTP referer header value. If not present, it is displayed as [-]. The HTTP User Agent (browser) typically sets this header to the URL of the link or containing page when making a request.
Example) http://www.example.com/webservices
19.User-Agent (user_agent)
HTTP user agent header value.
Example) Apache-httpClient/4.5.14 (java/17.0.9)
20.Version ID (version_id)
Version ID of the object being copied. If there is no value, it is displayed as [-]. (Currently we do not support version IDs.)
yes) -
21.Host ID (host_id)
Host machine id that made the request. It is recorded as an encrypted value.
Example) s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=
22.Protocol (protocol)
The requested Object Storage API Protocol. Supports [S3] API, [Swift] API.
Example) S3
23.Authentication type (authentication_type)
The type of request authentication used: [AuthHeader] for authentication headers, [QueryString] for query strings (pre-signed URLs), and [-] for unauthenticated requests.
Example) AuthHeader
24.Host header (host)
This is an endpoint for object storage.
Example) objectstorage.kr-central-2.kakaocloud.com