Guides

Remotely set up target device

Suggest Edits

What we are going to do

We will introduce about remote setting for easier use of Optimium. Optimium optimizes AI models perfectly on the designated device by running and profiling every layer operation. Thus, if you want to distribute your AI model on multiple devices, you should compile your model on each device. However, it is quite burdensome to setup all required settings for Optimium on each device. Moreover, you may not have enough computing power on your target device, like Raspberry Pi. To address this, we provide a remote setting.

ℹ️

The Optimium process is divided into two steps : 1) Optimize and 2) Deploy

The remote setting discussed in this topic matters only in the first step, 1) Optimize. When you serve the optimized AI model, you won't need to consider about the remote setting.

Requirements

  • A target device on which you want to optimize your AI model:
    • AMD64 or ARMv8 (or above)
    • For use of ARMv7, please contact us
  • The operating system (OS) of the target device should be either Linux or Android:
    • Linux version 18.04 or above
    • Android API level 26 (Android 8) or above
  • The target device must be connected to the host machine:
    • Linux target device must be connected via SSH
    • Android target device must be connected via either SSH or ADB

Terms and Concepts

In Optimium, we refer to two devices called the Target device and Host device.

The Target device is where you want to distribute and serve your AI model. It can be an edge device like a Raspberry Pi, NVIDIA Jetson, or a cloud instance like AWS EC2 or Azure Virtual Machine.

The Host device is where you run Optimium to optimize and compile the AI model. The optimization process is likely to consume a large amount of memory and computation; therefore, we highly recommend a powerful desktop. See Optimium for details.

In previous tutorials and the Quickstart, the Host device was equivalent to the Target device. We deliberately did not mention anything about remote setups for your ease of understanding, but we now highly recommend distinguishing between them.

Prepare for Remote Setting - Install Optimium Remote Server

On the target device, which is remote from the host device, only the Optimium Remote Server needs to be installed. (You still have to install Optimium and Optimium Runtime on your host device as mentioned in Optimium Setup)

The Optimium Remote Server installation process depends on whether your target device runs Linux or Android.

Linux Target Device

Run the following commands on your host machine where you downloaded the Optimium SDK (see Optimium Setup. Modify <port> and your-account@target-device in line 4 to match your target device's SSH settings. The command uses SSH and SCP to install Optimium Remote Server and will ask for your SSH password if needed.

# Host device
cd OPTIMIUM_SDK/optimium_runtime
./install_runtime.sh \
    --port <port> --remote <your-account@target-device> \
    all

After installing the Optimium Remote Server, update the PATH environment variable on your target device so that the Optimium Remote Server can work properly.

# Target device
export PATH="$PATH:$HOME/.local/bin"

Android Target device with adb

Run the following commands on your host machine where you downloaded the Optimium SDK (see Optimium Setup). Modify "serial" with the serial ID of the target device.

$ cd Optimium_SDK/optimium_runtime
$ ./install_runtime.sh --adb --serial <serialnum> bin native xnnpack

ℹ️

How do I get the serial of my Android device?

Open a terminal and type adb devices -l. The strings in the first row of the list are the serial ID of the devices.

If your Android device supports SSH, you may follow above Linux Target device section.


We have setup the remote setting on the target device. Now, it's time to optimize and serve an AI model on the target device for real . Let's get started next tutorials:

Updated 8 months ago