Introduction

Please follow the steps below if you are using SDK V1.61 or earlier version.
This modification will be incorporated in the next SDK version update.

Please delete the following part in “external/build/sample/docker/Dockerfile”.
6th line: ENV http_proxy=http://proxy.mei.co.jp:8080/ https_proxy=http://proxy.mei.co.jp:8080/
Please modify the following parts in “external/build/sample/docker/docker_volume/module_compile.sh”
4th line:
　Before：NUMPY_ARCHIVES=v1.19.4.tar.gz
　After：NUMPY_ARCHIVES=numpy-1.19.4.tar.gz
8th line:
　Before: OPENCV_ARCHIES=opencv-3.4.7.zip
　After: OPENCV_ARCHIVES=opencv-3.4.7.zip

Please follow the steps below if you are using SDK V1.70 or earlier version.
This modification will be incorporated in the next SDK version update.

Please delete the following part in “external/build/sample/docker/Dockerfile”.
4th line:
　Before：RUN python -m pip install Cython
　After：RUN python -m pip install Cython==0.29.36

Also, this tutorial refers to the SDK installation directory as ${SDK_DIR}.

Building external libraries is not supported. Please build the external library by referring to the information provided in this chapter. Please note that we cannot provide support even if you contact us.

Overview

Python module needs to be compiled when using external library with Python version AdamApp.

Here, we navigate how to build external library for Python from the environment building to module compilation.

External library used in this tutorial

NumPy 1.14.9
OpenCV 3.4.7
sqlite 3.40.0
Python 3.7.9
pycurl 7.45.2
OpenSSL 1.1.1

Establish Docker environment

Docker environment is built over the Ubuntu to which AdamAppSDK is installed.

OS used in this tutorial for installation

SDK v1.71 and earlier : Ubuntu 18.04.2 LTS (bionic)
SDK v1.80 or later : Ubuntu 22.04.3 LTS (Jammy Jellyfish)

AdamAppSDK development environment must be built in advance. (This tutorial does not include this step)

If you have already created an AdamAppSDK development environment using SDK ver.1.80 or later, there are some duplicate steps. Please omit unnecessary parts as appropriate. Also, please note that the Docker image created using .devcontainer/Dockerfile when building the AdamAppSDK development environment and the Docker image explained in this chapter are different Docker images.

In addition, as the installation require the use of apt-get command and such, prepare the Internet environment that allows you to run “apt-get”, and setup management setting such as root privilege for installation.

Reference: Docker official document

For VirtualBox environments, Docker compilation may fail on host-guest shared directories. If it fails, try working in a directory on Ubuntu.

Install the package

Install the package required to establish Docker environment.

$ sudo apt-get update
$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common \
    qemu-user-static

Add repository(When using the apt-key command)

The apt-key command is scheduled to be discontinued, although the date has not yet been determined. Please try the method that does not use the apt-key command described below. The method for adding a repository is just one example, so please consider alternative methods on your own.

~~Add apt repository for Ubuntu provided by Docker to your system.~~

~~Add Docker official PGP key to your system.~~

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
OK

~~Use -x option if you need a proxy server to connect to the Internet.~~

~~（Change URL and port number for the proxy server according to the proxy server you are using.)~~

~~$ curl -x~~ ~~http://proxy.example.com:8080/~~ ~~-fsSL~~ ~~https://download.docker.com/linux/ubuntu/gpg~~ ~~| sudo apt-key add -~~

~~Make sure the fingerprint looks like the following after adding PGP key.~~

$ sudo apt-key fingerprint 0EBFCD88
pub   rsa4096 2017-02-22 [SCEA]
      9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
uid           [ unknown ] Docker Release (CE deb) docker@docker.com
sub   rsa4096 2017-02-22 [S]

~~Then, add Docker official apt repository to your system.~~

$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

~~Add environment variable “http_proxy” and “https_proxy” as indicated below if you need a proxy server to connect to the Internet.~~

~~(Change URL and port number for the proxy server according to the proxy server you are using.)~~

~~$ sudo http_proxy=http://proxy.example.com:8080~~ ~~https_proxy=https://proxy.example.com:8080~~ ~~add-apt-repository "deb [arch=amd64]~~ https://download.docker.com/linux/ubuntu ~~$(lsb_release -cs) stable"~~

~~Check if you successfully added apt repository to “/etc/apt/sources.list”.~~

$ cat /etc/apt/sources.list
(Omitted)
deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
# deb-src [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable

Add repository(When not using the apt-key command)

The apt-key command is scheduled to be deprecated, so an example of an alternative method is provided. Please consider alternative methods on your own. Also, if you have already created a Docker image in an environment using the apt-key command, it may not work properly unless you create it again from the Docker container. please note.

Add apt repository for Ubuntu provided by Docker to your system.

Create a keyrings directory in the /etc/apt directory in advance.

$ mkdir /etc/apt/keyrings

Add Docker official PGP key to your system.

$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker-key.asc

Then, add Docker official apt repository to your system.

$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker-key.asc] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker execution file

Check if the system is up-to-date and then install Docker using the following apt-get command.

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

(Optional) Proxy setting of Docker

You may need proxy setting for the Docker if you need a proxy server to connect to the Internet.

Reference: Docker official document

Change the setting for Docker command

Change the setting for Docker command in order to make it available to general users.

$ sudo usermod -a -G docker $(whoami)

Log out and log in again to apply the setting change.

Confirm if Docker runs successfully

Check the installed Docker can be run in your system using the following command.

$ docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Create Docker image

Create a Docker image used to compile Python module

Edit Dockerfile

You need to create a file called Dockerfile in which the program to create an image is written when creating a Docker image.

We provide you with a Dockerfile template to create necessary image for compiling NumPy and OpenCV module with ease.

external/build/sample/docker/Dockerfile

FROM arm64v8/ubuntu:bionic

ADD qemu-aarch64-static /usr/bin
ADD exec_entry.sh /bin

RUN apt-get update \
 && apt-get install -y --no-install-recommends \
    build-essential gcc-8 g++-8 gfortran-8 python3.7 python3-distutils python3-setuptools python3.7-dev python3-pip cmake unzip \
 && apt-get -y clean && rm -rf /var/lib/apt/lists/*
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g++-8 \
 && update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-8 800 \
 && update-alternatives --install /usr/bin/python python /usr/bin/python3.7 800
RUN python -m pip install Cython==0.29.36
RUN chmod +x /bin/exec_entry.sh

ENTRYPOINT ["/bin/exec_entry.sh"]

※Do not change 1st, 3rd, 4th, 14th, and 16th line above when editing Dockerfile.

Add the following lines before

“RUN apt-get update \”

if you need a proxy server to connect to the Internet.

（Change URL and port number for the proxy server according to the proxy server you are using.)

ENV http_proxy=http://proxy.example.com:8080/ https_proxy=http://proxy.example.com:8080/

Build an image from a Dockerfile

Docker image is built within the directory where Dockerfile is placed.

$ cd ${SDK_DIR}/external/build/sample/docker/
$ docker build -t compile/ubuntu-rel:0.1 .

If the installation fails due to an error, please install qemu as described below.

Check the directory to see if a repository “compile/ubuntu-rel” appears when you finish building the image.

$ docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
compile/ubuntu-rel   0.1                 ca4409cf01fd        45 minutes ago      559MB

Make sure you see the following two files in the directory where Dockerfile is placed.

exec_entry.sh
qemu-aarch64-static

Do not delete these files as you will need them when creating Python module.

How to install qemu

qemu must be installed in the development environment in order to create container version additional apps. Describe how to install qemu. If you have completed creating the Docker image, there is no need to install qemu, so please proceed to the next chapter.

After restarting the PC, linux/arm64 may not appear in buildable architectures. In that case, please install qemu again by the same procedure.

install qemu

sudo apt update
sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages
sudo docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scripts

The third line also has a means to do the following:

sudo docker run --privileged --rm tonistiigi/binfmt --install all

The official document (Japanese) as of June 2023 guides here.

Docker Buildx | Docker ドキュメント (matsuand.github.io)

Preparing docker buildx

Add and reflect environment variables to use experimental features and BuildKit.
Open configuration file.
```
vi ~/.bashrc
```

Add the following to the end

export DOCKER_CLI_EXPERIMENTAL=enabled
export DOCKER_BUILDKIT=1

Save file.

Check if buildx is available.

docker --help | grep buildx

Output example

  buildx*     Docker Buildx (Docker Inc., v0.10.5)

Check the currently buildable architectures.

docker buildx ls

Output example

NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT                              PLATFORMS
default * docker
  default default         running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

If you see linux/arm64 in this output, success.

Compile Python module(Numpy, OpenCV)

Steps

Follow the steps below to compile Python module for AdamApp

Place source file in “${SDK_DIR}/external/build/sample/docker/docker_volume” directory
Write compilation program in “${SDK_DIR}/external/build/sample/docker/docker_volume/module_compile.sh”.
Run docker and proceed compilation.
The compiled module will be placed in “${SDK_DIR}/external/build/sample/docker/docker_volume/install_path” directory.

Here we take you through NumPy and OpenCV compilation as a tutorial.

The compilation fails if you work within shared folder over VirtualBox environment.
Create and move the folders to Ubuntu before you start.

Download source code

Download Numpy and OpenCV source code from the URLs below.

This tutorial uses Numpy 1.14.9, OpenCV 3.4.7.
Select the version to download according to the functions you use.

Numpy 1.14.9:
https://github.com/numpy/numpy/archive/v1.19.4.tar.gz
OpenCV 3.4.7:
https://github.com/opencv/opencv/archive/3.4.7.zip

Place the downloaded files to “${SDK_DIR}/external/build/sample/docker/docker_volume” directory.

Update compilation program

Write compilation program in the file: “${SDK_DIR}/external/build/sample/docker/docker_volume/module_compile.sh”.

Be aware the following points when updating the compilation program.

”${SDK_DIR}/external/build/sample/docker/docker_volume” will be mounted as “/home/docker” after you run docker.
Source files and compile directory are also placed in ”/home/docker”.
Install compiled modules to “/home/docker/install_path”.
You need NumPy module to compile OpenCV.

”module_compile.sh” is provided as a sample script to compile Numpy and OpenCV.

Here we proceed without changing “module_compile.sh”.

Compile

Compile NumPy module and OpenCV module using docker.

Run Docker in “${SDK_DIR}/external/build/sample/docker” directory.

$ docker run --rm -it --name aarch64-ubuntu -v `pwd`/docker_volume:/home/docker compile/ubuntu-rel:0.1

The compiled module is created in “external/build/sample/docker/docker_volume/install_path”.

Be patient while compiling OpenCV, it may take a while.
If a compilation error occurs with SDK V1.70 or earlier, please confirm that the external/build/sample/docker/Dockerfile described in the "Introduction" section has been corrected. If it is not corrected, start over from creating the Docker image.

Deployment in AdamApp

Deploy the modules to use them with AdamApp.

Here we take “${SDK_DIR}/src/adamapp-py/jpeg_app” as an example.

Deploy Numpy

NumPy module is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/site-packages/numpy-1.19.4-py3.7-linux-aarch64.egg/numpy

Copy and paste to AdamApp source directory without changing the directory structure.

${SDK_DIR}/src/adamapp-py/jpeg_app/python/site-packages

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/site-packages/numpy-1.19.4-py3.7-linux-aarch64.egg/numpy src/adamapp-py/jpeg_app/python/site-packages

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/jpeg_app/python/site-packages/numpy

Deploy OpenCV

OpenCV module is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/dist-packages/cv2

Copy and paste to AdamApp source directory without changing the directory structure.

${SDK_DIR}/src/adamapp-py/jpeg_app/python/site-packages

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/dist-packages/cv2 src/adamapp-py/jpeg_app/python/site-packages

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/jpeg_app/python/site-packages/cv2

Deploy both Open CV library and Python module as Open CV requires them both.

OpenCV library is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib

Copy “libopencv_world.so.3.4.7” and paste it to the following directory.

Please create manually after the external folder.

例）${SDK_DIR}/src/adamapp-py/jpeg_app/external/lib/aarch64-linux-gnu/

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libopencv_world.so.3.4.7 src/adamapp-py/jpeg_app/external/external/lib/aarch64-linux-gnu

Then create a symbolic link.

$ cd ${SDK_DIR}/src/adamapp-py/jpeg_app/external/lib/aarch64-linux-gnu/
$ ln -s libopencv_world.so.3.4.7 libopencv_world.so.3.4
$ ln -s libopencv_world.so.3.4 libopencv_world.so

Delete files no longer needed

AdamApp does not use (_pycache_), Python cash file.

We recommend that you delete the files to reduce the folder size.

$ cd ${SDK_DIR}/src/adamapp-py/yuv_yolo_app/python/site-packages/numpy/
$ find . -name *pycache* -print | xargs rm -rf

In addition, the information needed for debugging library is included in the OpenCV library you created.

We recommend that you delete this file to reduce the folder size as you do not need this to run the application.

$ cd ${SDK_DIR}/src/adamapp-py/yuv_yolo_app/external/lib/aarch64-linux-gnu/
$ aarch64-linux-gnu-strip -strip-debug libopencv_world.so.3.4.7

Build the sample application

Build the sample application "jpeg_app" and install it on the camera (eg, it can be installed from the green frame in the image below).

Please perform the build method according to each SDK version.

Open the app screen (red frame button in the image below), and if the jpeg image taken by the camera is displayed, it is successful.

Compile Python module(SQLite)

Numpy and OpenCV were compiled. As the next tutorial, we will compile SQLite.

Download source code

Download sqlite and Python source code from the URLs below.

This tutorial uses sqlite 3.40.0, Python 3.7.9.
Select the version to download according to the functions you use.

sqlite 3.40.0
https://www.sqlite.org/2022/sqlite-autoconf-3400000.tar.gz
Python 3.7.9
https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz

Place the downloaded files to “${SDK_DIR}/external/build/sample/docker/docker_volume” directory.

Update compilation instructions file

Put the compilation instructions in “${SDK_DIR}/external/build/sample/docker/docker_volume/module_compile.sh” file.

Please copy the content below.

#!/bin/sh -x

HOME_PATH=/home/docker

LIB_INSTALL_DIR=${HOME_PATH}/install_path

SQLITE_ARCHIVES=sqlite-autoconf-3400000.tar.gz
SQLITE_DIR=sqlite-autoconf-3400000
SQLITE_BUILD_DIR=sqlite-autoconf-3400000/build

PYTHONPATH=$PYTHONPATH:${NUMPY_SITE_PACKAGES}
export PYTHONPATH

PYTHON_ARCHIVES=Python-3.7.9.tgz
PYTHON_DIR=Python-3.7.9

## for sqlite
cd ${HOME_PATH}
tar zxf ${SQLITE_ARCHIVES}
mkdir -p ${SQLITE_BUILD_DIR}
cd ${SQLITE_DIR}
./configure --prefix=${LIB_INSTALL_DIR}
make
make install

## for python
cd ${HOME_PATH}
tar zxf ${PYTHON_ARCHIVES}
cd ${PYTHON_DIR}
./configure --prefix=${LIB_INSTALL_DIR}
make
make install

Compile

Compile sqlite module and Python module using docker.

Run Docker in “${SDK_DIR}/external/build/sample/docker” directory.

$ docker run --rm -it --name aarch64-ubuntu -v `pwd`/docker_volume:/home/docker compile/ubuntu-rel:0.1

The compiled module is created in “external/build/sample/docker/docker_volume/install_path”.

Deployment in AdamApp

Deploy the modules to use them with AdamApp.

Here we take “${SDK_DIR}/src/adamapp-py/sqlite_app” as an example.

An SD card is required for sqlite_app to work. Only cameras that can insert an SD card can be checked. Also, depending on the firmware of the camera, it may not work unless the SD card is formatted with ext4.

Deploy SQLite

SQLite module is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib

libsqlite3.so
libsqlite3.so.0
libsqlite3.so.0.8.6
libsqlite3.la
libsqlite3.a

Copy the above 5 files to the AdamApp source directory.

${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libsqlite3.so ${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu
$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libsqlite3.so.0 ${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu
$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libsqlite3.so.0.8.6 ${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu
$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libsqlite3.la ${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu
$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/libsqlite3.a ${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu/libsqlite3.so
${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu/libsqlite3.so.0
${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu/libsqlite3.so.0.8.6
${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu/libsqlite3.la
${SDK_DIR}/src/adamapp-py/sqlite_app/external/lib/aarch64-linux-gnu/libsqlite3.a

Deploy sqlite for Python

sqlite for Python module is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/sqlite3

Copy and paste to AdamApp source directory without changing the directory structure.

${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/sqlite3 ${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages/sqlite3

sqlite for Python requires not only modules but also libraries, so place them together.

The sqlite for Python library is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/lib-dynload

Copy “_sqlite3.cpython-37m-aarch64-linux-gnu.so” to the following directory.

${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages/sqlite3

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/lib-dynload/_sqlite3.cpython-37m-aarch64-linux-gnu.so ${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages/sqlite3

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages/sqlite3/_sqlite3.cpython-37m-aarch64-linux-gnu.so

Next, modify dbapi2.py so that the sqlite for Python library can be loaded when importing.

${SDK_DIR}/src/adamapp-py/sqlite_app/python/site-packages/sqlite3/dbapi2.py

Modify as follows.

Before) from _sqlite3 import *
↓
After) from sqlite3._sqlite3 import *

Build the sample application

Before building the sample app, format the SD card using the screen below. (Please confirm in advance whether the SD card can be formatted. All data on the SD card will be erased.)
Once formatting is complete, the SD card as ext (ext4) and press the Set button.

Build the sample application "sqlite_app" and install it on the camera (eg, it can be installed from the green frame in the image below).

Please perform the build method according to each SDK version.

Open the app screen (red frame button in the image below),

and if the following screen is displayed, it is successful.

Compile Python module(Pycurl)

SQLite was compiled. As the next tutorial, we will compile Pycurl.

Download source code

Download PyCurl and OpenSSL source code from the URLs below.

This tutorial uses pycurl 7.45.2, OpenSSL 1.1.1.
Select the version to download according to the functions you use.

pycurl 7.45.2
https://files.pythonhosted.org/packages/a8/af/24d3acfa76b867dbd8f1166853c18eefc890fc5da03a48672b38ea77ddae/pycurl-7.45.2.tar.gz
OpenSSL 1.1.1
https://github.com/openssl/openssl/archive/OpenSSL_1_1_1i.tar.gz

Place the downloaded files to “${SDK_DIR}/external/build/sample/docker/docker_volume” directory.

If the camera firmware is v3.10 or earlier, the camera uses OpenSSL 1.1.1, so it will work without placing OpenSSL. Camera firmware v3.20 and later has been updated to OpenSSL 3.0.12, so it will not work unless OpenSSL is placed. Please select and use the version of OpenSSL that matches your app environment.

Update compilation instructions file

Put the compilation instructions in “${SDK_DIR}/external/build/sample/docker/docker_volume/module_compile.sh” file.

Please copy the content below.

#!/bin/sh -x

HOME_PATH=/home/docker

OPENSSL_ARCHIVES=openssl-OpenSSL_1_1_1i.tar.gz
OPENSSL_DIR=openssl-OpenSSL_1_1_1i

PYCURL_ARCHIVES=pycurl-7.45.2.tar.gz
PYCURL_DIR=pycurl-7.45.2

apt update
apt install libcurl4-openssl-dev libssl-dev

cd ${HOME_PATH}
tar zxf ${OPENSSL_ARCHIVES}
cd ${OPENSSL_DIR}
./Configure linux-aarch64
make

export PYCURL_SSL_LIBRARY=openssl
export CPPFLAGS=-I${HOME_PATH}/${OPENSSL_DIR}/include/openssl
export LDFLAGS=-L${HOME_PATH}/${OPENSSL_DIR}

## for pycurl
cd ${HOME_PATH}
tar zxf ${PYCURL_ARCHIVES}
cd ${PYCURL_DIR}
python3.7 setup.py --with-openssl --openssl-dir=${HOME_PATH}/${OPENSSL_DIR} build

Compile

Compile pycurl module and OpenSSL module using docker.

Run Docker in “${SDK_DIR}/external/build/sample/docker” directory.

$ docker run --rm -it --name aarch64-ubuntu -v `pwd`/docker_volume:/home/docker compile/ubuntu-rel:0.1

The compiled module is created in “external/build/sample/docker/docker_volume/install_path”.

Deployment in AdamApp

Deploy the modules to use them with AdamApp.

Here we take “${SDK_DIR}/src/adamapp-py/hogehoge_app” as an example.

Deploy PyCurl

PyCurl module is installed in the following directory.

${SDK_DIR}/external/build/sample/docker/docker_volume/pycurl-7.45.2/build/lib.linux-aarch64-3.7

pycurl.cpython-37m-aarch64-linux-gnu.so

Copy the above file to the AdamApp source directory.

${SDK_DIR}/src/adamapp-py/hogehoge_app/python/site-packages

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/pycurl-7.45.2/build/lib.linux-aarch64-3.7/pycurl.cpython-37m-aarch64-linux-gnu.so ${SDK_DIR}/src/adamapp-py/hogehoge_app/python/site-packages

Then, the directory structure will be as follows.

${SDK_DIR}/src/adamapp-py/hogehoge_app/python/site-packages/pycurl.cpython-37m-aarch64-linux-gnu.so

Modify the source code to load PyCurl

Open “${SDK_DIR}/src/adamapp-py/hogehoge_app/python/pymain.py” and add as follows.

import os;
Add the following one line below.
import pycurl;

Build the sample application

Build the sample application "hogehoge_app" and install it on the camera (eg, it can be installed from the green frame in the image below).

Please perform the build method according to each SDK version.

Compile a Python module (Numba)

This article describes the steps to build Numba, which is used to speed up Python.

Download source code

Several modules are required to run Numba. Please download each source code from the URL below. Also, please select a version of each module that is compatible with Python 3.7.9.

Numba 0.56.4:
https://github.com/numba/numba/archive/refs/tags/0.56.4.zip
Numpy 1.19.4:
https://github.com/numpy/numpy/archive/v1.19.4.tar.gz
LLVM 11.1.0:
https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-11.1.0.tar.gz
llvmlite 0.39.1:
https://github.com/numba/llvmlite/archive/refs/tags/v0.39.1.tar.gz
importlib_metadata 6.7.0:
https://github.com/python/importlib_metadata/archive/refs/tags/v6.7.0.zip
typing_extensions 4.7.1:
https://github.com/python/typing_extensions/archive/refs/tags/4.7.1.zip
zipp 3.15.0:
https://github.com/jaraco/zipp/archive/refs/tags/v3.15.0.zip

Place the downloaded file in the following directory. ${SDK_DIR}/external/build/sample/docker/docker_volume

Update the compilation instructions file

The compilation procedure is described in the file below.
${SDK_DIR}/external/build/sample/docker/docker_volume/module_compile.sh

Please copy the content below.

#!/bin/sh -x

#----------
HOME_PATH=/home/docker
LIB_INSTALL_DIR=${HOME_PATH}/install_path
SITE_PACKAGES=${LIB_INSTALL_DIR}/lib/python3.7/site-packages

NUMPY_ARCHIVES=numpy-1.19.4.tar.gz
NUMPY_DIR=numpy-1.19.4

LLVM_ARCHIVES=llvm-project-llvmorg-11.1.0.tar.gz
LLVM_DIR=llvm-project-llvmorg-11.1.0
LLVM_INSTALL_DIR=${LIB_INSTALL_DIR}/llvm
LLVM_PATCH_SRC=${LLVMLITE_DIR}/conda-recipes
LLVM_PATCH_DST=${LLVM_DIR}/llvm

LLVMLITE_ARCHIVES=llvmlite-0.39.1.tar.gz
LLVMLITE_DIR=llvmlite-0.39.1

NUMBA_ARCHIVES=numba-0.56.4.zip
NUMBA_DIR=numba-0.56.4

#----------

# NumPy
cd ${HOME_PATH}
mkdir -p ${SITE_PACKAGES}
tar xf ${NUMPY_ARCHIVES}
cd ${NUMPY_DIR}
python setup.py build
PYTHONPATH=$PYTHONPATH:${SITE_PACKAGES}
export PYTHONPATH
python setup.py install --prefix=${LIB_INSTALL_DIR}


# LLVM / llvmlite
cd ${HOME_PATH}
tar xf ${LLVM_ARCHIVES}
tar xf ${LLVMLITE_ARCHIVES}

cd ${LLVM_PATCH_DST}
for patchFile in $(find ../../${LLVM_PATCH_SRC} -name "*.patch" -type f); do
	patch --dry-run -f -p1 -i ${patchFile}
	if [ $? -eq 0 ]; then
		patch -p1 -i ${patchFile}
	fi
done

LLVM_BUILD_SH="../../${LLVMLITE_DIR}/conda-recipes/llvmdev/build.sh"
mkdir -p ${LLVM_INSTALL_DIR}
export PREFIX=${LLVM_INSTALL_DIR} CPU_COUNT=2
sed -i -e "s/\(^LLVM_TARGETS_TO_BUILD=\).*/\1\$\{LLVM_TARGETS_TO_BUILD:-\"AArch64\"\}/" ${LLVM_BUILD_SH}
sed -i -e "/LLVM_ENABLE_ASSERTIONS/d" ${LLVM_BUILD_SH}
sed -i -e "/LLVM_EXPERIMENTAL_TARGETS_TO_BUILD/d" ${LLVM_BUILD_SH}
chmod +x ${LLVM_BUILD_SH}
${LLVM_BUILD_SH}
cd ../../

cd ${LLVMLITE_DIR}
export LLVM_CONFIG=${LLVM_INSTALL_DIR}/bin/llvm-config
python setup.py build
python setup.py install --prefix=${LIB_INSTALL_DIR}
cd ../


# numba
unzip ${NUMBA_ARCHIVES}
cd ${NUMBA_DIR}
export NUMBA_DISABLE_TBB=1
python setup.py build
python setup.py install --prefix=${LIB_INSTALL_DIR}
cd ../


# replace some libraries compatible with python3.7
cd ${HOME_PATH}
unzip importlib_metadata-6.7.0.zip
mv importlib_metadata-6.7.0/importlib_metadata ${SITE_PACKAGES}
unzip typing_extensions-4.7.1.zip
mv typing_extensions-4.7.1/src/typing_extensions.py ${SITE_PACKAGES}
unzip zipp-3.15.0.zip
mv zipp-3.15.0/zipp ${SITE_PACKAGES}


# cleanup
cd ${SITE_PACKAGES}
rm -f easy-install.pth setuptools.pth site.py
find . -maxdepth 2 -name numpy -type d -exec mv {} . \;
find . -maxdepth 2 -name numba -type d -exec mv {} . \;
rm -rf *.egg*
find . -name *pycache* -type d -print | xargs rm -rf
find . -name tests -type d -print | xargs rm -rf
find . -name *.so | xargs aarch64-linux-gnu-strip --strip-debug
cd ${HOME_PATH}

compile

Compile each module using Docker.
Run Docker in the directory below.
${SDK_DIR}/external/build/sample/docker

$ docker run --rm -it --name aarch64-ubuntu -v `pwd`/docker_volume:/home/docker compile/ubuntu-rel:0.1

Compiled modules are placed in the following directories.
${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/site-packages

LLVM will take some time to compile, so please wait for a while.
The total file size of the compiled module is approximately 60MByte.

Place in AdamApp

Place the created module so that it can be used by AdamApp.
The tutorial uses the following as an example.
${SDK_DIR}/src/adamapp-py/yuv_yolov5_app

Place the module

Copy the compiled module to the AdamApp source directory, keeping the directory structure as is.

$ cp -R ${SDK_DIR}/external/build/sample/docker/docker_volume/install_path/lib/python3.7/site-packages ${SDK_DIR}/src/adamapp-py/yuv_yolov5_app/python/

update pymain.py

Change the source code to be compatible with numba. Replace the attached file "pymain.py" with the file below.
\src\adamapp-py\yuv_yolov5_app\python\pymain.py

Build the sample app

Build the sample app "yuv_yolov5_app" and install it on the camera. Please perform the build method according to each SDK version. When you open the app screen, if there is a frame around the object (such as a person) on the screen, it is a success.

Please also refer to the operation explanation of the sample application.

If it fails, a message will be displayed on the application screen indicating that the application has failed to start. From now on, please use the message content and debugging tools to solve the problem and create an application.

Compile external library for Python

Introduction

Table of contents

Overview

External library used in this tutorial

Establish Docker environment

OS used in this tutorial for installation

Install the package

Add repository(When using the apt-key command)

Add repository(When not using the apt-key command)

Install Docker execution file

(Optional) Proxy setting of Docker

Change the setting for Docker command

Confirm if Docker runs successfully

Create Docker image

Edit Dockerfile

Build an image from a Dockerfile

How to install qemu

install qemu

Preparing docker buildx

Compile Python module(Numpy, OpenCV)

Steps

Download source code

Update compilation program

Compile

Deployment in AdamApp

Deploy Numpy

Deploy OpenCV

Delete files no longer needed

Build the sample application

Compile Python module(SQLite)

Download source code

Update compilation instructions file

Compile

Deployment in AdamApp

Deploy SQLite

Deploy sqlite for Python

Build the sample application

Compile Python module(Pycurl)

Download source code

Update compilation instructions file

Compile

Deployment in AdamApp

Deploy PyCurl

Modify the source code to load PyCurl

Build the sample application

Compile a Python module (Numba)

Download source code

Update the compilation instructions file

compile

Place in AdamApp

Place the module

Build the sample app