Skip to main content

integrate kubernetes cluster with object storage

Learn how to integrate object storage with kubernetes cluster to enable data interaction from kubernetes pods to object storage.

info

About this scenario​

This scenario explains how to utilize object storage from kubernetes pods by integrating kubernetes cluster with KakaoCloud object storage. Through this process, efficiently store and manage large-scale data in kubernetes environments.

Key topics include:

  • Configure secure authentication and access control using IAM access keys
  • Set up integration between kubernetes cluster and object storage

Before you start​

Prepare the authentication token and credentials required for object storage integration. Follow the steps below to complete the setup.

1. Create API authentication token​

Use a pre-issued access key to generate an API authentication token.
The API authentication token replaces the KakaoCloud account ID and password for authentication. Use the API authentication token with CLI or API to authenticate and access KakaoCloud services.

Issue API authentication token with access key
curl -X POST -i -H "Content-Type: application/json"  https://iam.kakaocloud.com/identity/v3/auth/tokens -d @<(cat << EOF  
{  
    "auth": {  
        "identity": {  
            "methods": [  
                "application_credential"  
            ],  
            "application_credential": {  
                "id": "${ACCESS_KEY}",  
                "secret": "${ACCESS_KEY_SECRET}"  
            }  
        }  
    }  
}  
EOF  
) | grep -i X-Subject-Token

Check the X-Subject-Token value from the output.

KeyDescription
“X-Subject-Token”API authentication token
info

The API authentication token is valid for 12 hours after issuance but may change or expire within this period. In such cases, a new token must be generated.

2. Create credential​

Generate credentials for using the S3 API. These credentials are required for secure integration with object storage.

Required information:

  • API authentication token: 1. Create API authentication token

  • Unique user ID: Found in [console] > [Account information]

  • Project ID: Found on the console main screen

    Issue credential
    curl -s -X POST -i https://iam.kakaocloud.com/identity/v3/users/${USER_ID}/credentials/OS-EC2 \  
    -H "Content-Type: application/json" \  
    -H "X-Auth-Token: ${API_TOKEN}" -d \  
    '{  
        "tenant_id": "${PROJECT_ID}"  
    }'

Check the access and secret values from the output.

KeyDescription
“access”Credential
“secret”Credential Secret
caution

Credentials are persistent once issued, so store them securely and manage them carefully to prevent exposure.

Getting started​

Start the step-by-step practice to create and integrate kubernetes cluster and object storage for data storage and utilization.

Step 1. Create kubernetes cluster​

A kubernetes cluster is the fundamental unit for deploying and managing containerized applications. You can create a cluster using Kubernetes Engine provided by KakaoCloud. For detailed instructions, refer to the Create kubernetes cluster documentation.

Step 2. Create Object Storage​

Object storage is optimized for storing and managing large-scale data in object (Key-Value) format. It is especially effective for processing unstructured data such as images, videos, and documents. Refer to the Create object storage documentation and create a bucket in KakaoCloud console > Object Storage.

Step 3. Create S3 secret in kubernetes cluster​

Use the credentials obtained in Step 2. Create credential to create an S3 secret in the kubernetes cluster. The S3 secret securely references authentication details for integration with object storage.

Create S3 secret
kubectl create secret generic s3-secret \  
--from-literal=credential="${access}" \  
--from-literal=credential-secret="${secret}" \  
--from-literal=region="kr-central-2" \  
--from-literal=endpoint-url="https://objectstorage.kr-central-2.kakaocloud.com"

Step 4. Deploy pod​

Write a YAML file for the pod referencing the S3 secret and deploy it to the kubernetes cluster to set up access to object storage.

s3-pod.yaml
apiVersion: v1  
kind: Pod  
metadata:  
  name: s3-access-pod  
spec:  
  containers:  
  - name: s3-container  
    image: ubuntu  
    env:  
    - name: AWS_ACCESS_KEY_ID  
      valueFrom:  
        secretKeyRef:  
          name: s3-secret  
          key: credential  
    - name: AWS_SECRET_ACCESS_KEY  
      valueFrom:  
        secretKeyRef:  
          name: s3-secret  
          key: credential-secret  
    - name: AWS_REGION  
      valueFrom:  
        secretKeyRef:  
          name: s3-secret  
          key: region  
    - name: AWS_ENDPOINT_URL  
      valueFrom:  
        secretKeyRef:  
          name: s3-secret  
          key: endpoint-url  
    command: ["/bin/sh", "-c"]  
    args:  
    - |  
      apt update && \  
      apt install -y curl unzip && \  
      curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \  
      unzip awscliv2.zip && \  
      ./aws/install && \  
      aws --version && \  
      sleep 3600
Deploy pod
kubectl apply -f s3-pod.yaml

Step 5. Verify object storage access​

Verify the connection to object storage from the deployed pod.

Verify object storage access from pod
kubectl exec -it s3-access-pod -- aws s3 ls

If the command outputs a list of files or directories stored in object storage, the connection is successfully established.