Kubeflow overview
Kubeflow on KakaoCloud is an open-source platform that helps you easily build and run machine learning workflows in a cloud-native environment based on Kubernetes Engine clusters. Kubeflow supports a wide range of machine learning libraries and frameworks that accelerate training processes, and provides various ML capabilities for running simple, portable, and scalable machine learning workloads on Kubernetes. In addition, by leveraging features such as Kubernetes cluster management and scaling, it enables stable and scalable operation of machine learning workflows.
- Kubeflow: A compound word of Kubernetes and ML flow, referring to an open-source platform provided to build and deploy ML workflows called pipelines. For more details, see the Kubeflow official documentation.
- Pipeline: In machine learning, a pipeline refers to a method of structuring the ML process in stages, connecting data processing and model training steps to efficiently transfer and process data.
Purpose and use cases
Training machine learning models requires multiple steps. When the size of training datasets exceeds hundreds of gigabytes, training becomes more complex and time-consuming. If users have to implement machine learning workflows themselves, they must configure Kubernetes clusters and install and manage machine learning frameworks directly, which can be cumbersome.
Kubeflow on KakaoCloud handles these complex processes on your behalf and helps users manage machine learning workloads more easily and efficiently. It also enables easier and more efficient execution of machine learning workloads, improving development productivity and allowing faster model development and deployment.
Kubeflow can be used in a wide range of fields where machine learning is essential, such as natural language processing, image processing, and recommendation systems.
Target users
Kubeflow can be used by users in various roles, including machine learning engineers, data scientists, developers, infrastructure engineers, and IT administrators.
| Target user | Description |
|---|---|
| Data scientist | Supports easy and fast machine learning modeling. Users with experience in data analysis and modeling can leverage Kubeflow to perform machine learning modeling more conveniently. |
| Machine learning engineer | Supports easy setup and management of machine learning frameworks and infrastructure. |
| Data engineer | Supports easy setup and management of big data processing and storage. Users with experience in data processing and storage technologies can use Kubeflow to perform data processing tasks more conveniently. |
| Cloud infrastructure engineer | Can easily build and manage machine learning workloads based on Kubernetes clusters. |
Features
Kubeflow is a free, open-source, Kubernetes-based machine learning platform that provides tools for distributed machine learning tasks and enables configuration of machine learning workflows across diverse environments. This allows machine learning workloads to be easily scaled and managed. In addition, it supports a variety of machine learning libraries and frameworks and provides multiple components for building workflows.
Kubernetes-based
- Operates based on Kubernetes Engine
- Supports various Kubernetes features and efficient resource management capabilities
Support for diverse machine learning libraries and frameworks
- Supports a wide range of machine learning libraries and frameworks such as TensorFlow, PyTorch, and XGBoost, and provides a platform for easy deployment and management
- Enables easy setup and execution of various machine learning workflows through Kubeflow Pipelines
Automated machine learning workflows
- Automates machine learning workflows to support rapid model development and deployment
- Provides automated workflows for each stage, including data preprocessing, model training, and model deployment
Efficiency and scalability
- Enables fast and stable deployment and scaling in cloud environments, while Kubernetes provides automated scheduling, logging, and monitoring
- Defines each stage in a standardized way and provides all necessary capabilities to automate the entire machine learning model workflow, improving development and operational efficiency
- All components are provided as open source, allowing users to modify or customize code as needed to build solutions optimized for their own machine learning workflows
Resource security and access control
- Allows assignment of namespaces to users based on tasks and roles, and optimizes resource management through quota allocation
- By adding required roles and owners as group members through group namespaces (kbm-g), permissions can be easily managed at the group level
Getting started
Detailed usage guides for Kubeflow are provided in How-to Guides. If you are new to KakaoCloud, refer to Getting started with KakaoCloud.