Access Logging
Access logging records detailed information about requests performed on Object Storage buckets. You can use it to check which requests occurred in a bucket and for security auditing, access auditing, and client request analysis.
To use access logging, configure a source bucket where logs are generated and a destination bucket where logs are stored. Logs are stored as objects in the destination bucket, and the required permissions must be granted to the destination bucket for logs to be transferred properly.
Bucket role permissions for log transfer
In access logging, the bucket where logs are generated is the source bucket, and the bucket where logs are stored is the destination bucket. Logs from the source bucket are stored in the destination bucket using the credentials of the Object Storage Agent service account.
For logs to be stored properly in the destination bucket, the Object Storage Agent service account must have the storage.buckets.get and storage.buckets.update permissions for the Object Storage role.
The Object Storage Agent account is a service account and has the IAM Project Member role. When a bucket is created, the IAM Project Member group is granted the storage editor role by default, allowing logs to be transferred without additional configuration.
If the Project Member group is removed from the destination bucket permissions, the Object Storage Agent account in that group loses the storage editor permission and cannot store logs. In this case, you must add the Object Storage Agent service account with the storage editor role or higher.
When using an SSE-KMS encrypted destination bucket
If the destination bucket is encrypted with a KMS user key (SSE-KMS), the Object Storage Agent service account must have permission to access that KMS user key for logs to be stored properly.
Without access permission, log storage may fail. Add the Object Storage Agent service account as a key user in KMS user key access control.
Configure access logging
Use the access logging settings to store event logs generated in a bucket in another bucket.
-
Go to KakaoCloud Console > Storage > Object Storage.
-
Select a bucket from the General bucket menu.
-
In the Properties list, click [Configure access logging].
-
Configure access logging in the popup, and click [Save].
Category Description Access logging Enable or disable access logging Destination bucket Select the bucket where logs will be stored Destination prefix (optional) Path within the destination bucket Log object key format Format of stored log object keys
-SimplePrefix: [DestinationPrefix][YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString]
-PartitionedDateSource: [DestinationPrefix][ProjectID]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/[YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString]
- The time format is based on the Object Storage event timestamp.Log object key example Example log object name generated using the selected object key format
When logs are transferred from the source bucket to the destination bucket, log entries are recorded and stored within a few hours after a request is made. However, if the bucket logging status changes, the changes are not applied immediately and logs are stored gradually over time. Log transfer may also be delayed or may not be fully guaranteed depending on the storage usage environment and system status. Therefore, you must verify the log transfer results.
Log object key format
The keys of access log objects stored in the destination bucket can be generated in the following formats.
SimplePrefix:DestinationPrefixYYYY-MM-DD-hh-mm-ss-UniqueStringPartitionedDateSource:DestinationPrefixSourceAccountId/SourceRegion/SourceBucket/YYYY/MM/DD/YYYY-MM-DD-hh-mm-ss-UniqueString
Each value in the object key format is as follows.
YYYY,MM,DD,hh,mm,ss: year, month, day, hour, minute, second (Coordinated Universal Time (UTC))DestinationPrefix: destination prefixProjectID: project IDSourceRegion: region nameSourceBucket: source bucketUniqueString: object key identifier string used to prevent log overwrites
Log record fields
Access logs record each request as one record. Fields in each record are separated by a space, and if data does not exist or is unknown, it is indicated by the - symbol.
The following describes the log record fields recorded in access logs.
Domain ID
The customer's domain ID.
e.g.) 327373ec52974577a79a5e26b26c27e9
Project ID
The project ID under the customer's domain.
e.g.) ca7f6c731a004091a32d4eb97ec17271
Bucket name
The name of the bucket that processes the request.
e.g.) Kakao-bucket
Bucket owner ID
The owner ID of the source bucket.
e.g.) 54ba02ba408d4968a35686e48db85ea8
Time
The time the request was made to the bucket. It uses the [%d/%b/%Y:%H:%M:%S %z] format and is displayed in Coordinated Universal Time (UTC).
- %d: two-digit day
- %b: abbreviated month name (e.g., May)
- %Y: four-digit year
- %H: two-digit hour
- %M: two-digit minute
- %S: two-digit second
- %z: UTC offset
e.g.) 16/May/2024:08:20:05 +0000
Remote IP
The IP address of the client that made the request.
e.g.) 127.0.0.1
User ID
The requester ID. For public access without a separate authentication token, it is indicated as -.
e.g.) 0e26ca49d2ca4bbfbd85e5901545c796
Request ID
An ID generated to identify the request.
e.g.) tx000008b923132a7716acd-0065795106-8fb2f-kr-central-2
Operation
The operations listed here are declared as [REST.{HTTP_method}.{resource_type}]. Objects deleted by lifecycle policies are not logged.
e.g.) REST.POST.OBJECT
Object key
The requested object key.
e.g.) /Image/kakaocloud/ryan.jpg
Request URI
The Request-URI of the HTTP request message.
e.g.) /v1/1b5e24ba80104e9f9aecd2bcfeb7da2/object-reg-test-1/mulit-object?uploads
HTTP status
The HTTP status code of the response.
e.g.) 200
Error code
The Object Storage error code. If there is no error, it is indicated as -.
(Only recorded for S3 API calls.)
e.g.) -
Request body size
The number of request bytes received.
e.g.) 2662992
Response body size
The number of response bytes sent, excluding HTTP protocol overhead.
e.g.) 5432290
Object size
The total size of the object.
e.g.) 7452918
Total time
The time consumed by Object Storage to process the user request. It is expressed in milliseconds. Measured from the time the request bytes are received to the time the final response byte is sent.
e.g.) 253.507608ms
HTTP referer
The HTTP referer header value. If there is no value, it is indicated as -.
HTTP user agents (browsers) usually set this header to the URL of the linked or embedded page when making a request.
e.g.) http://www.example.com/webservices
User-Agent
The HTTP User-Agent header value.
e.g.) Apache-httpClient/4.5.14 (java/17.0.9)
Version ID
The version ID of the object being copied. If there is no value, it is indicated as -.
(Version ID is currently not supported.)
e.g.) -
Host ID
The ID of the host machine that performed the request. It is recorded as an encrypted value.
e.g.) s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=
Protocol
The Object Storage API protocol used for the request. S3 API and Swift API are supported.
e.g.) S3
Authentication type
The request authentication type used. Header authentication is indicated as AuthHeader, query string (presigned URL) as QueryString, and unauthenticated requests as -.
e.g.) AuthHeader
Host header
The Object Storage endpoint.
e.g.) objectstorage.kr-central-2.kakaocloud.com