3 posts tagged with "analytics"

On November 19, 2024, KakaoCloud Pub/Sub was finally released as a generally available (GA) service! 🎉

During the beta service period, we made various updates with the goals of strengthening stability and improving usability, and by reflecting valuable customer feedback, we are now able to provide stronger and more advanced features. With the generally available Pub/Sub service, you can now manage and process large-scale data more efficiently. In today's post, we briefly introduce the key features of Pub/Sub and major improvements included in the GA release.

What is Pub/Sub?​

KakaoCloud Pub/Sub is a serverless message queue service designed for high-volume events and data analytics. You can classify and manage messages or events through topics, and use subscriptions so that subscribers can receive and process messages published to topics.

Users can simplify messaging structures and optimize real-time data processing by using Pub/Sub. In particular, it provides solutions optimized for various business environments such as event notifications between applications, data streaming, and asynchronous job processing. In this way, KakaoCloud Pub/Sub provides a powerful foundation for implementing an efficient and scalable messaging system that can meet diverse business requirements.

Major improvements and notes for the Pub/Sub GA release​

This GA release of Pub/Sub includes the following improvements to increase stability and usability.

1. New feature added

• The Object Storage subscription type has been added. This lets you send messages to an Object Storage bucket and stably store and use large volumes of messages.

2. Feature improvements and stability enhancements

• Enhanced API and SDK support
  • Support has been strengthened so that topics and subscriptions can be created and deleted using APIs and SDKs. For details, see the API Reference and SDK Reference in the technical documentation.
• More granular subscription status values
  • Status values have been subdivided further, improving message management efficiency.
• Cloud Trail and Alert Center event items added
  • Monitoring and notification management have been further strengthened.
• Expanded monitoring items
  • Monitoring items for topics and subscriptions have been expanded, allowing message processing status to be checked and managed in more detail.

3. SLA applied

• With the GA release, an SLA (Service Level Agreement) has been applied, enabling a more stable and reliable service.

4. Notice of transition to paid service

• As of November 20, 2024, Pub/Sub has transitioned to a paid service, and usage fees apply. Customers already using Pub/Sub should review the pricing policy in detail.

Closing​

Experience business innovation with Pub/Sub​

With KakaoCloud Pub/Sub, you can maximize data processing efficiency and automate various workflows to raise business competitiveness to the next level. For details about the service, see the Pub/Sub technical documentation. If you have any questions, please contact us anytime through KakaoCloud 1:1 inquiry.

Thank you.

KakaoCloud's new service, Advanced Managed Kafka, has been released.

Advanced Managed Kafka is a fully managed service designed to let users benefit from real-time data streaming while minimizing the operational burden of Kafka.

In today's environment, where data is collected and analyzed in real time, many companies adopt Apache Kafka as a data streaming tool. Kafka has excellent performance and flexibility, but it is a complex system that requires advanced configuration and continuous monitoring. Operating and managing Kafka directly requires significant technical burden and time. KakaoCloud developed Advanced Managed Kafka, a fully managed service designed to minimize the operational burden for Kafka users while allowing them to benefit from real-time data streaming.

Now let's take a closer look at the key features and characteristics of Advanced Managed Kafka.

What is Advanced Managed Kafka?​

Advanced Managed Kafka is a cloud-based service that lets you easily operate Kafka clusters from creation to management. It is suitable for applications that require real-time data streaming, and users can build a stable message queue and streaming environment without complex Kafka configuration.

Key features of Advanced Managed Kafka​

The basic concepts of Advanced Managed Kafka are Cluster and Broker.

A cluster is a core component of the Kafka environment. Advanced Managed Kafka automatically allocates and manages the required resources through simple settings, greatly reducing the complexity of Kafka cluster operations. A broker is a component of a cluster and is responsible for storing and delivering messages.

In Advanced Managed Kafka, broker management can optimize cluster performance and increase data availability.

1. Easy Kafka environment setup

When creating a cluster through Advanced Managed Kafka, you only need to enter simple required information such as cluster name, region and network, number of broker nodes, and volume size. The cluster is then automatically deployed according to the resources configured by the user and prepared for operation.

2. Cluster scaling

When data throughput increases, you may need to scale the cluster to improve processing performance. Advanced Managed Kafka helps users easily increase the number of broker nodes.

3. Volume expansion

When storage space becomes insufficient due to data growth, you may need to expand the volume. Advanced Managed Kafka helps users easily expand volume size.

4. Real-time monitoring and fast response

Advanced Managed Kafka provides features for monitoring key performance metrics of clusters and brokers in real time. Through metrics such as broker, memory, and disk usage and network I/O, you can check cluster status and performance at a glance and improve operational stability by managing clusters before issues occur.

Closing​

Advanced Managed Kafka provides various tools to stably manage real-time streaming data and flexibly operate clusters and brokers. This allows services where data flow is important to secure both high performance and stability.

Select Advanced Managed Kafka in the KakaoCloud console and easily build a Kafka environment. You can find more details in KakaoCloud's How-to Guides documentation.

You may also want to refer to the tutorial Message processing through Kafka, which covers the process of building a Kafka environment after creating a cluster and sending and receiving messages.

Thank you.

Gartner, a global information technology (IT) research and consulting company, researches and announces Data & Analytics (D&A) trends every year.
According to this year's Gartner report (Gartner Identifies the Top 10 Data and Analytics Trends for 2023), data and analytics teams must do more than manage data resources and generate insights from them. Beyond simply collecting massive amounts of data, they are required to collect the right data with the right tools at the right time and derive business insights from it. To do this, the report suggests that enterprise data and analytics teams should follow trends such as value optimization, data sharing, observability, data and analytics sustainability, and data fabric.

To continue keeping pace with rapidly evolving data analytics trends, KakaoCloud added the new Dataflow cluster type to Hadoop Eco in November 2023. Previously, Hadoop Eco provided Core Hadoop, HBase, and Trino types. With the addition of Dataflow clusters, data collection and analysis through Hadoop, Kafka, Druid, and Superset is now possible.
Dataflow provided by Apache Beam is one of the unified batch and streaming data processing models widely chosen by users around the world. Dataflow is a fully managed open-source framework optimized for streaming data analytics that minimizes latency, processing time, and cost through autoscaling and batch processing, and it supports a wide range of frameworks (Flink, Spark, and more) and multiple languages.

With the newly added Dataflow clusters in KakaoCloud Hadoop Eco, users can experience the following benefits.

• More efficient data collection and analysis: Efficiently collect data through Kafka and analyze data in real time using Druid and Superset.
• Various analysis tools provided: Visualize data and perform diverse analysis tasks through Druid and Superset.
• Scalability and high availability: Standard (Single) and high availability (HA) types are provided with cluster operation stability in mind.
  In Standard (Single) mode, one master node instance runs one resource manager and one name node, making it suitable for small-scale jobs. In the high availability (HA) type, three master node instances are provided, and the resource manager and name node run in HA mode. Uninterrupted work is possible even when three master nodes are created or rebooted.

Selecting a Dataflow cluster

Try Dataflow, the unified batch and streaming data processing model provided by Apache Beam, in KakaoCloud Hadoop Eco.
Thank you.