Use by instance type

Use GPU instance

To use a GPU-type instance, you must install the appropriate driver. You can either create an instance using a GPU-optimized OS image with drivers pre-installed, or install the drivers separately after creating an instance with a base image. The steps for installing GPU drivers per OS and utilizing the GPU are as follows.

info

This guide is based on the Ubuntu 20.04 image with NVIDIA A100 currently offered by KakaoCloud.

Step 1. Install NVIDIA driver

Install the NVIDIA driver. The recommended driver and CUDA versions are:

GPU type	NVIDIA version	CUDA version
NVIDIA A100	450.80.02 or higher	CUDA Toolkit 11.1 or higher

info

• For detailed instructions on downloading NVIDIA drivers, refer to the NVIDIA official site > Driver Downloads.
• For details on installation and configuration, see the NVIDIA official site > NVIDIA Driver Installation Quickstart Guide.

1. Run the following command to confirm whether an NVIDIA device is present in the instance:

   Detect NVIDIA device

   lspci | grep -i NVIDIA

2. Check the list of available drivers.

   • If the installed driver is outdated, run apt update -y to refresh the list.

   • If Command ‘ubuntu-drivers’ not found appears, install it with:

     sudo apt install ubuntu-drivers-common
   Check available driver versions

   ubuntu-drivers devices
   Example output

   ubuntu-drivers devices
   ...
   driver   : nvidia-driver-515 - distro non-free recommended
   ...

3. Install the desired driver version.

   Install driver

   sudo apt install nvidia-driver-470

4. Reboot the system.

   Reboot command

   sudo reboot

5. Verify the installed driver.

   Verify driver

   nvidia-smi
   Example output

   +-----------------------------------------------------------------------------+
   | NVIDIA-SMI 470.141.03   Driver Version: 470.141.03   CUDA Version: 11.4     |
   ...
   +-----------------------------------------------------------------------------+

Step 2. Install NVIDIA CUDA Toolkit

Install the NVIDIA CUDA Toolkit.

Delete toolkit when reinstalling CUDA Toolkit

If reinstalling CUDA Toolkit, prepare the environment as follows:

1. Remove previous CUDA configurations.

   Remove CUDA config

   sudo rm -rf /usr/local/cuda*

2. Remove the following lines from ~/.bashrc or /etc/profile if present:

   Remove existing exports

   export PATH=$PATH:/usr/local/cuda-11.4/bin
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.4/lib64
   export CUDADIR=/usr/local/cuda-11.4

3. Confirm nvcc is no longer available:

   Check nvcc

   nvcc -V

   Expected: Command 'nvcc' not found

Install CUDA Toolkit

1. Select the desired version from the NVIDIA official site > CUDA Toolkit Archive. Copy the Base Installer commands.

2. Download the installer:

   Download installer

   wget https://developer.download.nvidia.com/compute/cuda/11.4.0/local_installers/cuda_11.4.0_470.42.01_linux.run

3. Run the installer:

   Run installer

   sudo sh cuda_11.4.0_470.42.01_linux.run

4. Use arrow keys to select Continue, then press Enter.

5. Enter accept and press Enter.

6. Uncheck Driver with the Space key, select Install, then press Enter.

   • If an existing installation is detected, choose Upgrade all.

7. On successful installation, you will see a confirmation message.

8. Add environment variables:

   Add CUDA environment variables

   sudo sh -c "echo 'export PATH=$PATH:/usr/local/cuda-11.4/bin' >> /etc/profile"
   sudo sh -c "echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.4/lib64' >> /etc/profile"
   sudo sh -c "echo 'export CUDADIR=/usr/local/cuda-11.4' >> /etc/profile"
   source /etc/profile

9. Confirm installation:

   Verify CUDA Toolkit

   nvcc -V

Use NPU instance

To use an NPU-type instance, you must install the appropriate driver. You can use an NPU-specific OS image with pre-installed drivers, or install them manually after creating an instance with a base image. Follow the steps below to install the NPU driver in a Linux environment.

info

• This guide is based on the Ubuntu 20.04 image with FuriosaAI WARBOY.
• APT repository configuration from FuriosaAI is required. Refer to FuriosaAI Docs > Driver, Firmware, Runtime Installation Guide.
• For technical support, contact the FuriosaAI Support Center.

Step 1. Install FuriosaAI driver

Install the FuriosaAI NPU driver. Recommended versions:

NPU type	Driver version	SDK version
FuriosaAI Warboy	1.7 or higher	0.9.1

1. Confirm the presence of FuriosaAI hardware.

   Detect FuriosaAI device

   lspci -nn | grep 1200

   or

   Alternative detection

   sudo update-pciids
   lspci | grep FuriosaAI

2. Install the driver.

   Install driver

   sudo apt install furiosa-driver-warboy

3. Install runtime libraries and tools.

   Install libraries and tools

   sudo apt install furiosa-libnux furiosa-toolkit

4. Lock package versions.

   Hold package versions

   sudo apt-mark hold furiosa-driver-warboy furiosa-libhal-warboy furiosa-libcompiler furiosa-libnux furiosa-toolkit libonnxruntime

5. Confirm driver installation.

   Verify driver

   furiosactl info
   Example output

   +------+--------+----------------+-------+--------+--------------+
   | NPU  | Name   | Firmware       | Temp. | Power  | PCI-BDF      |
   +------+--------+----------------+-------+--------+--------------+
   | npu0 | warboy | 1.6.0, c1bebfd |  52°C | 2.52 W | 0000:00:05.0 |
   +------+--------+----------------+-------+--------+--------------+

Step 2. Install FuriosaAI Python SDK

Install the FuriosaAI Python SDK.
We recommend using a virtual environment (venv, pyenv, pipenv, conda, etc.). This guide uses Miniconda.

info

• For detailed installation steps, refer to FuriosaAI Docs > Python SDK Installation Guide.
• For broader documentation, see FuriosaAI Docs > NPU and SDK documentation.

1. Install Miniconda.

   Install Miniconda

   wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
   SHELL=bash sh ./Miniconda3-latest-Linux-x86_64.sh
   source ~/.bashrc
   conda --version

2. Create and activate a Python environment.

   Create environment

   conda create -n my-env python=3.9
   conda activate my-env
   python --version

3. Install the SDK.

   Install SDK

   pip install --upgrade pip setuptools wheel
   pip install 'furiosa-sdk[full]'

4. Verify the SDK.

   Verify SDK

   furiosa --version
   Example output

   0.9.1-release (rev: a240782)

Step 3. Run the FuriosaAI Python SDK example

Run an inference example using the installed SDK.

1. Clone the example repository.

   Clone repository

   git clone https://github.com/furiosa-ai/furiosa-sdk --depth 1

2. Install required libraries.

   Install libraries

   cd furiosa-sdk/examples/inferences
   pip install -r requirements.txt

3. Run the example.

   Run example

     (my-env)$ ./image_classify.py ../assets/images/car.jpg
    libfuriosa_hal.so --- v0.11.0, built @ 43c901f
    INFO:furiosa.common.native:loaded native library libnux.so (0.9.1 d91490fa8)
    Loading and compiling the model /home/ubuntu/furiosa-sdk/examples/inferences/../assets/quantized_models/imagenet_224x224_mobilenet_v1_uint8_quantization-aware-trained_dm_1.0_without_softmax.tflite
    Saving the compilation log into /home/ubuntu/.local/state/furiosa/logs/compile-20230516064013-fzjjxx.log
    Using furiosa-compiler 0.9.1 (rev: d91490fa8 built at 2023-04-19T13:49:26Z)
    2023-05-16T06:40:13.769147Z  INFO nux::npu: Npu (npu0pe0-1) is being initialized
    2023-05-16T06:40:13.773685Z  INFO nux: NuxInner create with pes: [PeId(0)]
   
    ...
   
    Prediction elapsed 0.00 secs
    [Top 5 scores:]
    sports car: 155
    pickup: 152
    car wheel: 148
    convertible: 148
    racer: 143
    2023-05-16T06:41:00.568028Z  INFO nux::npu: NPU (npu0pe0-1) has been destroyed
    2023-05-16T06:41:00.568453Z  INFO nux::capi: session has been destroyed