Access Logging
Access Logging is a feature that records requests performed on Object Storage buckets for use in security auditing and access analysis.
Understand and use access logging
When the Access Logging feature is enabled, detailed records of requests performed on an Object Storage bucket are logged. These logs can be used for purposes such as security and access auditing or analyzing client requests to the bucket.
For details on how to enable access logging, refer to Configure access logging.
Access logs record each request as a single entry, with each field separated by a space. If a value does not exist or is unknown, it is represented by a -.
Detailed explanations of each field can be found in Log record fields.
Bucket role permissions for log transfer
Logs are stored from the source bucket to the destination bucket using credentials of the Object Storage Agent service account.
To ensure logs are stored properly, the destination bucket must grant the Object Storage Agent account the storage.buckets.get and storage.buckets.update permissions for the Object Storage role.
The Object Storage Agent account is a service account with the IAM project member role.
When a bucket is created, the IAM project member group is granted the storage editor role by default, allowing logs to be transferred without additional configuration.
When logs are transferred from the source bucket to the destination bucket, entries are recorded and stored within a few hours after a request is made. However, if the logging status of the bucket is changed, updates are not applied immediately but reflected gradually over time. Log transfer may also be delayed or not guaranteed depending on usage and system conditions. Therefore, it is important to verify log transfer results.
If the project member group is removed from the destination bucket's permissions, the Object Storage Agent account in that group will lose its storage editor role and will not be able to store logs. In this case, you must add the Object Storage Agent service account with at least the storage editor role as shown below.
Add Object Storage Agent role
Object key format of log
Access logs are created using the following object key formats:
- SimplePrefix:
DestinationPrefixYYYY-MM-DD-hh-mm-ss-UniqueString - PartitionedDateSource:
DestinationPrefixSourceAccountId/SourceRegion/SourceBucket/YYYY/MM/DD/YYYY-MM-DD-hh-mm-ss-UniqueString
- Object key format details:
YYYY,MM,DD,hh,mm,ss: year, month, day, hour, minute, second (in UTC)DestinationPrefix: destination prefixProjectID: project IDSourceRegion: region nameSourceBucket: source bucketUniqueString: identifier string for the object key
UniqueString prevents logs from being overwritten.
Log record fields
The following describes the log record fields recorded in the access logs.
1. Domain ID (domain_id)
The customer’s domain ID value.
```
e.g.) 327373ec52974577a79a5e26b26c27e9
```
2. Project ID (project_id)
The project ID under the customer’s domain.
```
e.g.) ca7f6c731a004091a32d4eb97ec17271
```
3. Bucket name (bucket)
The name of the bucket that processed the request.
```
e.g.) Kakao-bucket
```
4. Bucket owner ID (bucket_owner)
The owner ID of the source bucket.
```
e.g.) 54ba02ba408d4968a35686e48db85ea8
```
5. Time (time)
The time the request was made to the bucket. Uses the format [%d/%B/%Y:%H:%M:%S %z]
- %d: two-digit day
- %B: abbreviated month name (e.g., May)
- %Y: four-digit year
- %H: two-digit hour
- %M: two-digit minute
- %S: two-digit second
- %z: UTC offset
The time is recorded in UTC.
``` e.g.) 16/May/2024:08:20:05 +0000 ```
6. Remote IP (remote_ip)
The IP address of the client making the request.
```
e.g.) 127.0.0.1
```
7. User ID (user_id)
The requester ID. Public access without an auth
8. request_id (request_id)
An ID generated to identify the request.
e.g.) tx000008b923132a7716acd-0065795106-8fb2f-kr-central-2
9. operation (operation)
The operation is expressed as [REST.{HTTP_method}.{resource_type}].
Objects deleted through lifecycle policy are not logged.
e.g.) REST.POST.OBJECT
10. key (key)
The key of the object being requested.
e.g.) /Image/kakaocloud/ryan.jpg
11. request_uri (request_uri)
The Request-URI from the HTTP request message.
e.g.) /v1/1b5e24ba80104e9f9aecd2bcfeb7da2/object-reg-test-1/mulit-object?uploads
12. http_status (http_status)
The HTTP status code of the response.
e.g.) 200
13. error_code (error_code)
Object Storage-specific error code.
If there is no error, it is recorded as -.
(Only recorded for S3 API calls.)
e.g.) -
14. request_body_size (request_body_size)
Number of bytes received in the request.
e.g.) 2662992
15. response_body_size (response_body_size)
Number of response bytes sent, excluding HTTP protocol overhead.
e.g.) 5432290
16. object_size (object_size)
Total size of the object.
e.g.) 7452918
17. total_time (total_time)
The time consumed by Object Storage to process the request, in milliseconds.
Measured from the time the request bytes are received to the time the final response byte is sent.
e.g.) 253.507608ms
18. http_referer (http_referer)
Value of the HTTP referer header.
If none, recorded as -.
Browsers typically set this to the URL of the referring or embedding page.
e.g.) http://www.example.com/webservices
19. user_agent (user_agent)
Value of the HTTP User-Agent header.
e.g.) Apache-httpClient/4.5.14 (java/17.0.9)
20. version_id (version_id)
Version ID of the object being copied.
If not available, recorded as -.
(Currently, versioning is not supported.)
e.g.) -
21. host_id (host_id)
ID of the host machine that processed the request, recorded in encrypted form.
e.g.) s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=
22. protocol (protocol)
The API protocol used for the Object Storage request.
Supports both [S3] and [Swift] APIs.
e.g.) S3
23. authentication_type (authentication_type)
Type of authentication used in the request.
AuthHeader for header-based authentication, QueryString for presigned URL, and - for unauthenticated requests.
e.g.) AuthHeader
24. host (host)
The endpoint of Object Storage.
e.g.) objectstorage.kr-central-2.kakaocloud.com