Table of contents
Preparation
Please refer here for the environment construction procedure.
Conversion
Please refer to the following page for model conversion.
AI model convert tool: Tensorflow
AI model convert: ONNX(PyTorch)
Evaluate AI model
The CV tool does not include a mechanism to evaluate the inference time, accuracy, etc. of the converted model.
Run the converted model on the i-PRO camera and evaluate it.
Inference time
Measure the time before and after the inference execution API Adam_AI_RunNet() call
precision
Evaluate the accuracy using the data obtained by the API / Adam_AI_GetOutput () that acquires the data of the output layer of the model.
The CV tool includes a sample app that allows you to obtain inference time and output layer data for the converted model.
Preparing for the evaluation
Use the Chrome extension's ADAM OPERATION UI for evaluation.
For details on how to install, refer here.
Also, make sure you have an i-PRO network camera that the app can install on.
DnnSdApp
Install the app
First, copy the DnnSdApp package (DnnSdApp_V0_5_ambaCV2X5X.ext) in the container to the host PC.
[Work Directory]
Any directory
$ cd [Work Directory] $ sudo docker run -it --rm -v $(pwd):/work [image name] /bin/bash $ cp /home/cvtool/app/DnnSdApp_V0_5_ambaCV2X5X.ext /work
Launch a browser and access the detailed setting
Set [Basic] - [SD memory card] - [Operation Mode] - [SD memory card] and [Ext. software mode] to "On".
If you do not want to use the SD card, please select "Not use".
When uploading a model, use the method of transferring it on a TFTP server.
Move to the Ext. software and install DnnSdApp.
Change settings to match your rating model
Configure various settings with ADAM OPERATION UI.
layernamein: Input layer name
layernameout: Output layer name (separated by comma for multiple settings)
DnnSdApp may not work properly, when “/” is contained in layernamein or layernameout.
NETNAME: Model name
TftpServerIP:TFTP server address where models are stored
*Set if SD card is not used
ChannelNum:Channels of model
ImgHeight:Height of input image
ImgWidth:Width of input image
PixelFormat:Pixel format of model
Prepare evaluation images
Compress the images to be used for evaluation (dnn.tar.gz) .
Follow the folder structure below, either jpeg or mp4 only can be used.
tar cvzf dnn.tar.gz dnn
Folder configuration | Remarks | ||
---|---|---|---|
dnn/ | test_jpeg/ | yyy1.jpg | jpeg placement directory, file names are arbitrary File extension: ".jpg", ".jpeg", ".JPG", ".JPEG"
|
yyy2.jpg | |||
: | |||
test_mp4/
| zzz1.mp4 | mp4 placement directory, file name to be deployed is arbitrary File ectension: ".mp4" | |
zzz2.mp4 | |||
: |
Upload images to DnnSdApp
Open the app screen and upload the image data.
If the size of the image data (dnn .tar.gz) is larger than 70MB, place the dnn folder directly on the SD card.
[SD Card]/dnn_sd_app/dnn/~~
Upload the model file to the app
Upload the model file.
If you don't use an SD card, use a TFTP server to upload your model.
Store the model file on a TFTP server (the same IP as the one set in AppPref), and then click the "Send" button to transfer the model to the camera.
Run the app
After placing the model and image, click the "Start" button to start execution.
Download the results
Once the run is complete, you can get the result file from the Download button.
SsdSdApp
The operation is similar to DnnSdApp.
Please replace the folder name with "SSD" ⇒ "DNN".