Supported Environments

Optimium Runtime supports Linux and Android OS with X64 and Arm64 architectures.

OS \ Arch	AMD64(x64)	Arm64
Linux	✅	✅
Android	❌	✅

We support Arm64 v8-A and above. If your device uses Arm64 v7, please contact us.

Requirements

Linux

Ubuntu 18.04 or above
- OpenMP (KMP, not GOMP): 10 or above
- libstdc++: 8 or above
- glibc: 2.27 or above
Python: 3.6 ~ 3.12 (Optimium Runtime for Python)
Java: 11 or above (Optimium Runtime for Kotlin)

🚧
Not tested on non-debian based Linux distros.

Android

Android API: 26 (Android 8) or above

ℹ️
Built with minSdkVer == 26, compileSdkVer == 33 and android-ndk-r25b.

Components

Optimium Runtime is composed of various components. User can selectively install Optimium Runtime according to their applications, however, we strongly recommend installing all components by using all.

Depending on the languages, some components may be inaccessible or have different names. Please check below installation guide for each language. Before that, available components are listed here ⬇️

Components	Description
`all`	Install every Optimium Runtime components
`bin`	Install Optimium model executor and remote server. Required when using Remote optimization feature of Optimium
`dev`	Install header files and development tools support files
`lib`	Install core Optimium Runtime components
`native`	Install Optimium Runtime extension for Native(CPU) devices. This extension is mandatory.
`xnnpack`	Install Optimium Runtime extension for XNNPACK

Install

Optimium Runtime provides bindings for various languages. You can choose according to your preference.

📘
This document assumes that Optimium installation path is defined as OPTIMIUM_SDK_ROOT environment variable.
If you have not exported the variable, recommend to append export OPTIMIUM_SDK_PATH=<your-install-path> in your .bashrc file.
In order to run the optimized model on your target device, you need to download a runtime on target device. Since the runtime is included inside Optimium SDK, we’ve created a folder named “(Optional) Resource limited target device” to prevent users from manually extracting runtime from Optimium SDK. When you use this archive which only contains Optimium Runtime rather than Optimium SDK, '$OPTIMIUM_SDK_ROOT' must be replaced to '$OPTIMIUM_RT_ROOT'.

Before Installation

You should install the prerequisite below.

This is not required for Android, as OpenMP is already included in the distribution.

# If your libomp5 version is below 10, use libomp5-10 instead.
sudo apt-get install libomp5

# Make symbolic link to prevent error.
MULTIARCH=$(dpkg-architecture -q DEB_TARGET_MULTIARCH)
sudo ln -s "/usr/lib/$MULTIARCH/libomp.so.5" "/usr/lib/$MULTIARCH/libomp.so"

The supported languages are listed below:

C++ with automated scripts (Host PC or "local")

1️⃣ To install Optimium Runtime, type command below.

$ cd Optimium_SDK/runtime
$ ./install_runtime.sh all

install_runtime.sh will extract packages to $HOME/.local folder on your host device.

Additionally, You can install Optimium Runtime for other architectures to facilitate cross-compilation.

$ cd Optimium_SDK/runtime
$ ./install_runtime.sh --cross arm64-linux dev

2️⃣ Configure your environment.

Configure your PATH environment variable to run Optimium Runtime properly.

export PATH="$PATH:$HOME/.local/bin"

IMPORTANT! If you're using Android, please configure your environment using below.

export PATH="$PATH:/data/local/tmp/.local/bin"
export LD_LIBRARY_PATH="/data/local/tmp/.local/lib/aarch64-linux-android"

C++ with automated scripts (target device or "remote")

1️⃣ To install Optimium Runtime, type command below.

$ cd Optimium_SDK/runtime
$ ./install_runtime.sh \
  --remote your-account@your-device \
  all

install_runtime.sh will extract packages to $HOME/.local folder on your target device.

The script uses SSH and SCP to install Optimium Runtime on your target device and will ask for your SSH password if needed.

2️⃣ Configure your environment.

Configure your PATH environment variable to run Optimium Runtime properly.

export PATH="$PATH:$HOME/.local/bin"

C++ with manual installation

1️⃣ Get Optimium Runtime.

If you want to get the runtime without using the automated installation script, you can note the archive that contains all components at $OPTIMIUM_SDK_DIR/runtime/cpp/archives/optimium-runtime-<version>-<arch>-<os>.tar.gz.

Copy it into your working directory.

cp Optimium_SDK/runtime/cpp/archives/optimium-runtime-<version>-<arch>-<os>.tar.gz optimium-runtime.tar.gz

2️⃣ Unzip the archive.

You can install Optimium Runtime in any path you want. In this document, it is assumed that you install the runtime at $HOME/.local.

mkdir -p $HOME/.local
tar xzf optimium-runtime.tar.gz -C $HOME/.local

3️⃣ Configure your environment.

Configure your PATH environment variable to run Optimium Runtime properly.

export PATH="$PATH:$HOME/.local/bin"

Python

Install Optimium Runtime via pip.
```
$ cd Optimium_SDK/
$ pip install -f "runtime/python" optimium-runtime[all]
```
- To install necessary components, simply add extras to the package name.
- The provided extras are listed below:
  - optimium-runtime[all]
    - Combined package of lib, native, xnnpack and language binding.
  - optimium-runtime[native] (optimium-runtime-native)
    - Same as native component.
  - optimium-runtime[xnnpack] (optimium-runtime-xnnpack)
    - Same as xnnpack component.

Kotlin

1️⃣ Add local Maven repository to your settings.gradle (or settings.gradle.kts) file.

// Write path to Optimium SDK path or
// use System.getenv("OPTIMIUM_SDK_ROOT")
def OPTIMIUM_SDK_ROOT = "..."

dependencyResolutionManagement {
    repositories {
        // ... other repositories
        maven {
            name "Optimium"
            url "${OPTIMIUM_SDK_ROOT}/runtime/kotlin"
        }
    }
}

// Write path to Optimium SDK path or
// use System.getenv("OPTIMIUM_SDK_ROOT")
val OPTIMIUM_SDK_ROOT = "..."

dependencyResolutionManagement {
    repositories {
        // ... other repositories
        maven {
            name = "Optimium"
            url = uri("${OPTIMIUM_SDK_ROOT}/runtime/kotlin")
        }
    }
}

2️⃣ Add dependency to Optimium Runtime in your project.

dependencies {
    def OPTIMIUM_RUNTIME_VERSION = "0.3.2"

    // For JVM platform
    implementation "com.enerzai.optimium.runtime:jvm:${OPTIMIUM_RUNTIME_VERSION}"
    runtimeOnly "com.enerzai.optimium.runtime:jvm:${OPTIMIUM_RUNTIME_VERSION}:x64-linux"

    // Native extension
    runtimeOnly "com.enerzai.optimium.runtime:jvm-native:${OPTIMIUM_RUNTIME_VERSION}:x64-linux"

    // XNNPack extension
    runtimeOnly "com.enerzai.optimium.runtime:jvm-xnnpack:${OPTIMIUM_RUNTIME_VERSION}:x64-linux"

    // For Android platform
    implementation "com.enerzai.optimium.runtime:android:${OPTIMIUM_RUNTIME_VERSION}"

    // Native extension
    runtimeOnly "com.enerzai.optimium.runtime:android-native:${OPTIMIUM_RUNTIME_VERSION}"

    // XNNPack extension
    runtimeOnly "com.enerzai.optimium.runtime:android-xnnpack:${OPTIMIUM_RUNTIME_VERSION}"
}

dependencies {
    val OPTIMIUM_RUNTIME_VERSION = "0.3.2"

    // For JVM platform
    implementation("com.enerzai.optimium.runtime:jvm:${OPTIMIUM_RUNTIME_VERSION}")
    runtimeOnly("com.enerzai.optimium.runtime:jvm:${OPTIMIUM_RUNTIME_VERSION}:x64-linux")

    // Native extension
    runtimeOnly("com.enerzai.optimium.runtime:jvm-native:${OPTIMIUM_RUNTIME_VERSION}:x64-linux")

    // XNNPack extension
    runtimeOnly("com.enerzai.optimium.runtime:jvm-xnnpack:${OPTIMIUM_RUNTIME_VERSION}:x64-linux")


    // For Android platform
    implementation("com.enerzai.optimium.runtime:android:${OPTIMIUM_RUNTIME_VERSION}")

    // Native extension
    runtimeOnly("com.enerzai.optimium.runtime:android-native:${OPTIMIUM_RUNTIME_VERSION}")

    // XNNPack extension
    runtimeOnly("com.enerzai.optimium.runtime:android-xnnpack:${OPTIMIUM_RUNTIME_VERSION}")
}