2 posts tagged with "cloud-trail"

In production environments, logs are reference data for troubleshooting incidents and reviewing security. However, storing logs is not enough. To analyze operational issues and identify causes, you must be able to query the data quickly with the conditions you need. For logs that are repeatedly used for audits and diagnostics, it is especially important to consider the storage location, file structure, and query method from the beginning.

The two newly added tutorials explain how to use Data Catalog and Data Query to query and analyze operational logs stored in Object Storage with SQL.

Both documents use the same data analysis architecture, but they focus on different operational scenarios. One covers security auditing and change tracking based on user activity and resource change history. The other covers network diagnostics based on DNS query flows inside a VPC. Accordingly, the target logs are Cloud Trail logs and DNS resolver query logs.

Operational log storage -> Object Storage -> Data Catalog -> Data Query

Although the logs being analyzed are different, the workflow is the same: store logs in Object Storage, configure table metadata in Data Catalog, and query the data with SQL in Data Query.

This post looks at when each log is useful and how to analyze operational logs with SQL.

Cloud Trail logs: Reference data for checking resource change history​

Cloud Trail records user activity and resource operation history in KakaoCloud as events. For example, you can check when a specific user logged in, which resources were created or modified, and which service generated an event. When these logs are connected to Data Query, you can answer common security audit and history tracking questions with SQL.

• Which user changed which resource on a specific date?
• Is there any operation history for a specific service or IP address?
• Did create, update, or delete events occur for a specific resource?

The Query Cloud Trail logs with Data Query tutorial explains how to store Cloud Trail logs in gz format in Object Storage and configure Data Catalog and Data Query based on the project_event and domain_event paths. In this workflow, it is important to use partition columns such as date_id and hour_id as query conditions so that you only query the required period.

Because Cloud Trail logs are used for security audits and change tracking, it is better to narrow the query by conditions such as when, from which service, by whom, and for which resource, rather than scanning all logs at once.

DNS resolver query logs: Check DNS queries and responses inside a VPC​

DNS resolver query logs record DNS query and response information generated inside a VPC. You can check which domains an application queried, whether responses were normal, and whether failed responses were concentrated in a specific time period. With Data Query, you can answer operational questions such as:

• Which domains were queried most on a specific date?
• During which time periods were non-NOERROR responses concentrated?
• Did a specific VPC query a specific domain unusually often?
• Are DNS queries with long response times recurring?

The Query DNS resolver query logs with Data Query tutorial configures tables based on the Object Storage path structure KCLogs/{region-name}/{year=yyyy/month=mm/day=dd} used by DNS resolver query logs. It then shows how to synchronize the year, month, and day partitions in Data Query and aggregate query counts or failed response counts by domain.

DNS logs are useful not only for network incident analysis, but also for checking external dependencies of internal services, unexpected domain queries, and repeated failed responses. If Cloud Trail shows the history of users and resource operations, DNS resolver query logs show how applications inside a VPC perform name resolution.

Why the two tutorials use the same pattern​

Cloud Trail logs and DNS resolver query logs have different characteristics, but operators handle them in similar ways.

First, logs are stored in Object Storage. Next, metadata is configured in Data Catalog based on file paths and partition structures. Finally, Data Query uses SQL to query the logs with conditions such as specific periods, services, users, domains, and response codes.

After this common pattern is established, you can reuse the same flow of storage location, metadata configuration, and SQL querying even when the analysis target changes.

This structure helps manage operational log analysis in a consistent way. Instead of learning different query languages and storage structures for each separate tool, you can standardize the query flow around Object Storage, Data Catalog, and Data Query.

How to start operational log analysis​

If you need to retain operational logs for a long time and query them when necessary, we recommend reviewing both tutorials together. For example, when an incident occurs during a specific time period, you can check resource change history with Cloud Trail logs and also review DNS query failures or response delays from the same time period with DNS resolver query logs. When logs with different characteristics can be queried in the same way, it becomes easier to interpret individual events in a broader operational context.

KakaoCloud technical documentation provides various tutorials based on practical operational scenarios. Use the following documents to learn how to store operational logs, configure them in a queryable form, and analyze them step by step with the conditions you need.

👉 Query Cloud Trail logs with Data Query
👉 Query DNS resolver query logs with Data Query

Using the cloud is like operating a virtual building with dozens of keys. 🔐
If it is not clear who can enter which room and which doors they can open, confusion quickly follows.
Deciding who receives these keys and under what conditions is exactly what IAM (Identity and Access Management) does. In other words, IAM is a service that grants only the permissions needed according to each user's role, helping manage resources efficiently and reduce unnecessary access.

However, for those encountering IAM for the first time, the concept may feel somewhat complex and burdensome.
To help users understand and use KakaoCloud IAM more accurately, the content planning team created a four-part onboarding video series.
In this post, we briefly summarize the key content of each video.

🎬 Part 1. Getting started with IAM - Concepts and basic structure​

The first video in the IAM onboarding series introduces the basic concepts of IAM and the structure of projects and organizations.
Even users new to IAM can easily understand the overall IAM structure through this video. Like looking at a city map, view the big picture of what permissions should be assigned to each area.

🎬 Part 2. IAM groups and service accounts - Improving user management efficiency​

Part 2 introduces two features you must know to operate IAM more conveniently and systematically: IAM groups and service accounts.

• IAM groups are a useful feature that groups users who need the same permissions into one user group and configures the required permissions all at once. For example, if you group users by teams such as development, operations, or marketing and configure the required permissions for each team at the group level, when a new team member joins, permissions are automatically granted simply by adding the member to the group. This enables much more efficient user management.
  
• Service accounts are non-user accounts used by applications or automation scripts to access or control resources within a project, rather than actual IAM user accounts. They can issue API tokens and call KakaoCloud APIs instead of using IAM user accounts.

By using these two features appropriately, user management and system permission settings can be operated more systematically and securely. See the video for details.

🎬 Part 3. Tracking IAM change history with Cloud Trail​

Initial IAM setup is important, but continuously checking and managing change history is also important. In Part 3, we introduce how to use KakaoCloud Cloud Trail to track who changed which IAM settings and when, at the event level.

🎬 Part 4. Reviewing IAM operational best practices​

The final video introduces five best practices for operating IAM stably. Check whether all five operational tips below are applied in your organization.

• Grant only the minimum permissions needed, without unnecessary permissions.
• Use Cloud Trail to regularly check change history.
• Regularly review and clean up departed-user and dormant accounts.
• Clearly separate user accounts and service accounts for operations.
• Integrate with Alert Center to quickly detect and respond to anomalies.

How was it?
IAM is more than a simple permission management tool. It is an important standard for securely protecting resources in an organization and clearly separating roles and responsibilities.
If you understand IAM's basic structure and operating methods well, you can continue providing stable and reliable services even in complex cloud environments.

If you want to learn more about KakaoCloud IAM, see the links below. Thank you :)

• 🔗 Go to KakaoCloud IAM documentation
  
• 👉 Start KakaoCloud now