Kubeflow overview
Kubeflow on KakaoCloud is an open-source platform for easily building and running machine learning workflows in a cloud-native environment based on Kubernetes Engine clusters. Kubeflow supports a wide range of machine learning libraries and frameworks that accelerate training, and provides capabilities for running portable and scalable machine learning workloads on Kubernetes. It also uses Kubernetes cluster management and scaling features to help you scale and operate machine learning workflows reliably.
- Kubeflow: An open-source platform for deploying and operating machine learning workflows on Kubernetes. For more information, see the Kubeflow documentation.
- Pipeline: A workflow that connects machine learning stages such as data preprocessing and model training to transfer and process data efficiently.
Purpose and use cases
Training machine learning models requires multiple steps. When the size of training datasets exceeds hundreds of gigabytes, training becomes more complex and time-consuming. If users have to implement machine learning workflows themselves, they must configure Kubernetes clusters and install and manage machine learning frameworks directly, which can be cumbersome.
Kubeflow on KakaoCloud simplifies these processes so that users can run machine learning workloads more easily and manage them efficiently. This improves development productivity and shortens the time required to develop and deploy models.
Kubeflow can be used in a wide range of fields where machine learning is essential, such as natural language processing, image processing, and recommendation systems.
Target users
Kubeflow can be used by users in various roles, including machine learning engineers, data scientists, developers, infrastructure engineers, and IT administrators.
| Target user | Description |
|---|---|
| Data scientist | Supports easy and fast machine learning modeling. Users with experience in data analysis and modeling can leverage Kubeflow to perform machine learning modeling more conveniently. |
| Machine learning engineer | Supports easy setup and management of machine learning frameworks and infrastructure. |
| Data engineer | Supports easy setup and management of big data processing and storage. Users with experience in data processing and storage technologies can use Kubeflow to perform data processing tasks more conveniently. |
| Cloud infrastructure engineer | Can easily build and manage machine learning workloads based on Kubernetes clusters. |
Features
Kubeflow is a free, open-source, Kubernetes-based machine learning platform that provides tools for distributed machine learning tasks and enables configuration of machine learning workflows across diverse environments. This allows machine learning workloads to be easily scaled and managed. In addition, it supports a variety of machine learning libraries and frameworks and provides multiple components for building workflows.
Kubernetes-based
- Operates based on Kubernetes Engine
- Supports various Kubernetes features and efficient resource management capabilities
Support for diverse machine learning libraries and frameworks
- Supports a wide range of machine learning libraries and frameworks such as TensorFlow, PyTorch, and XGBoost, and provides a platform for easy deployment and management
- Enables easy setup and execution of various machine learning workflows through Kubeflow Pipelines
Automated machine learning workflows
- Automates machine learning workflows to support rapid model development and deployment
- Provides automated workflows for each stage, including data preprocessing, model training, and model deployment
Efficiency and scalability
- Enables fast and stable deployment and scaling in cloud environments, while Kubernetes provides automated scheduling, logging, and monitoring
- Defines each stage in a standardized way and provides all necessary capabilities to automate the entire machine learning model workflow, improving development and operational efficiency
- All components are provided as open source, allowing users to modify or customize code as needed to build solutions optimized for their own machine learning workflows
Resource security and access control
- Allows assignment of namespaces to users based on tasks and roles, and optimizes resource management through quota allocation
- By adding required roles and owners as group members through group namespaces (kbm-g), permissions can be easily managed at the group level
Getting started
Detailed usage guides for Kubeflow are provided in How-to Guides. If you are new to KakaoCloud, refer to Getting started with KakaoCloud.