Basic and important implementation points
- Yoshitaka Fukushima (福島 佳孝)
- 1 YC image usage
- 2 Utilize AI accelerator
- 3 Sending data to external devices
- 4 Run camera firmware API (CGI) from app
- 5 Utilize camera’s internal RAM and ROM
- 6 Utilize SD memory card
- 7 WEB UI implementation
- 8 Log implementation
- 9 Use of multiple AI models and multiple apps
- 10 Keep-alive monitoring
- 11 CV2x and CV52 camera app compatibility and how to distinguish SoC
- 12 About development in Python
YC image usage
Available resolution list for app
The list of YC image resolutions available to the app varies depending on the camera firmware settings.
Support upside-down image
When using the image rotation and upside-down settings of the CV2x camera, the YC images the app can get are not automatically flipped, and the app has to implement the flipping process.
In addition, for some CV2x cameras (box type, dome type), regardless of the upside-down setting, the YC images that the app can get are inverted by default due to the orientation of the image sensor. Please refer to the following page to implement the inversion process after getting the YC images.
This is implemented in the sample applications web_template_app, yuv_dnn_app, yuv_pose_ap, yuv_ssd_app, yuv_yolo_app, yuv_yolov5_app, and yuv_yolov8_app included in SDK v2.01 or later.
360-degree fisheye camera
Depending on the image capture mode configuration, app can use the desired images such as fisheye images or de-warped images in the app.
Multi-directional camera
For multi-directional cameras, when installing the app, select the camera number (No. 1 to No. 4) to install the app on. When using ADAM_DevYcVideo_Create, the YC images got by the app will automatically be those from that camera number. When using ADAM_DevRawVideo_Create, it is possible to specify the image from a specific camera number or automatically, depending on the argument.
If you want to display video from a camera number on the web UI of the app, for example, please use Adam_GetSensorChannel to get the camera number where the app is installed, and then use the CGI.
（ex. /cgi-bin/mjpeg?ch=2, /cgi-bin/mjpeg?ch=3, /cgi-bin/mjpeg?ch=4)
This is implemented in the sample applications web_template_app.
The frame rate of the YC image for multi-directional camera is limited.
2M / 4M : max. 15 fps
5M / 6M / 4K : max. 7.5 fps
Utilize AI accelerator
・AI accelerator is available only in AdamApp for ambaCV2X,CV5X model.
・Tool(cvtool) is not included in AdamAppSDK. Please contact i-PRO Co., Ltd. if you need one.
Accelerator (CVFlow) for AI inference process is embedded in ambaCV2X,CV5X models. AdamApp can use the accelerator through a dedicated Adam API. A binary file for accelerator is needed as an input for API. Create a binary file and built it into AdamApp by the following sequence.
Convert a pre-trained AI model into a binary file for accelerator.
Create AdamApp package in accord to here in this document. Place the binary file generated in step 1 under data/cnn directory.
A tool (cvtool) to be used to process the step1 above is provided by Adam. Please see here for how to use cvtool.
Example of AI processing sequence
YC image frame rate and application processing time
If an application requests YC images at 10 fps, for example, the firmware will notify the application of the callback at 10 fps (approximately every 100 msec).
The application's callback function must complete its callback function within this interval.
If the application's previous callback function has not yet completed when the callback is notified (if the application's processing is delayed), the next notification will be made when the previous callback function ends. In this case, the image data application gets will be old data. If this delay continues, the firmware will initialize the image data in memory to reset the delay.
To check if your application is running slower, refer to the logs of the sample application yuv_ssd_app (in SDK v2.10 or later). The difference between the timestamp m_timeStamp of the YC image and the system time when the callback function is executed in the application is output. There is no problem as long as this difference does not continue to increase.
Perform AI processing when camera firmware event is triggered
Instead of performing image analysis on all YC images, you can use ADAM_DevInputAlarm_Create to have the app receive VMD or terminal alarm events detected by the camera and then process them for a fixed period of time (this is implemented in the sample app web_template_app), or you can use ADAM_DevPtzPresetPositionState_Create to receive the trigger that PTZ camera moves to a preset position.
ADAM_DevInputAlarm_Create is not supported for python.
Sending data to external devices
When event is generated in the app, app can use the SDK to trigger alarm on camera firmware, or the app can communicate directly with external devices.
ADAM_Alarm_Notify
The following camera firmware functions can be triggered.
ex.
TCP alarm notification, HTTP alarm notification
ref. https://dev-partner-en.i-pro.com/space/TPFAQEN/1039798079/TCP+Alarm%2FHTTP+Alarme-mail notification
Video recording to SD card
JPEG Image transmission by FTP/SFTP
Terminal alarm output
*This SDK is implemented in the sample applications alarm_sample_app and web_template_app.
ADAM_MetaDataStream_SetRaw
Data can be sent as RTSP metadata channel. Software that utilizes this data needs to be developed by our partners.
Also, by using a specified data format, it can be integrated with i-PRO Active Guard and various VMS automatically.
https://dev-partner-en.i-pro.com/space/TPFAQEN/1051271527/i-PRO+Active+Guard+integration+guide
*This SDK is implemented in the sample applications meta_stream_app and web_template_app.
ADAM_AdditionalInfo_Set
Data can be added to the RTP extension header for H.264, H.265, and audio transmission. In case of JPEG, COM segment is used. It is neccessary for partner to develop client software.
*This SDK is implemented in the sample application additional_info_sample_app.
Other protocol
Partners can freely implement protocols. The sample app mqtt_sample implements MQTT.
Run camera firmware API (CGI) from app
It is possible to run CGI from the application to the HTTP port of the camera firmware as local communication. Since user authentication is required, the camera's user name and password must be implemented as application configuration. The camera's HTTP port number setting, which is required for local communication, can be got by ADAM_GetHttpInfo.
CGI documentation: https://i-pro.com/products_and_solutions/en/surveillance/learning-and-support/device-integration/i-pro-camera
About CGI details, please contact us from the support page of each region.https://i-pro.com/products_and_solutions/en/surveillance/contact-us
Utilize camera’s internal RAM and ROM
Available resources depending on the camera model.
ref. https://dev-partner-en.i-pro.com/space/TPFAQEN/701433474/Installation+conditions+for+applications
Storing data to ROM (flash memory)
ADAM_GetAppDataDirPath provides the directory to store data into flash memory inside the camera. The data will be kept even if the camera is turned off and on.
Because there is a limit to the number of times that a flash memory can be written to, if you want to save data detected from video, you need to send it to an SD card or external device and save it there.
When temporarily storing into RAM is enough, ADAM_GetAppTmpDirPath provides the direcltory on RAM, and you can use it as long as it does not exceed the RAM limit.
Resource Monitor tool in SDK 2.01 or later allows you to check the status of access to flash memory. Please check during development to see if there is any unnecessary access.
ref: https://dev-partner-en.i-pro.com/space/TPFAQEN/1039828049/Resource+Monitor
If the used ROM size exceeds the limit, camera's firmware will stop the app. In this case, “Illegal termination (ROM capacity limit (actual capacity))” will be displayed in the camera's web browser control log.
Storing data to RAM
It is possible to temporarily store data in the directory got by ADAM_GetAppTmpDirPath.
In this case, please make sure not to exceed the upper limit.
If the used RAM size exceeds the limit, camera's firmware will stop the app. In this case, “Illegal termination (RAM capacity limit)” will be displayed in the camera's web browser control log.
How to get resource information
During development, you can check it using the Resource Monitor tool included in the SDK.
If you want to get it within the app, you can do so in one of the following ways.
Adam WebAPI getAppResourceInfo
/proc/[pid]/status, /proc/[pid]/statm
ADAM_GetRomRamSize *Supported in SDK v2.01 or later
Utilize SD memory card
To access SD memory card from the app, set [On] for [Basic] - [SD memory card] - [Ext. software mode] on camera's web.
ADAM_GetSdCardPath provides the directory.
With external library SQLite, you can also create a database on the SD memory card. This is implemented in the sample app sqlite_app. To use SQLite, please change the camera's [SD memory card file system] setting to [ext] and format the SD card in advance.
When [FAT] is used for [SD Memory Card File System Format] setting, the recording function to the SD memory card by the camera firmware also works, but this cannot be guaranteed.
When [ext] is used, the recording function to the SD memory card by the camera firmware does not work.
There is a limit to the number of times you can write to an SD memory card. For details, please refer to the manufacturer's information.
WEB UI implementation
You can freely implement WEB UI in your app and display it from the camera's WEB browser (e.g. displaying MJPEG and detection bounding box, changing app settings, etc.).
WEB UI is also implemented in various sample apps, and web_template_app includes functions that are generally required for apps.
If license activation is required for the app, this must be implemented by the partner. SDK provides camera's MAC address and serial number.
Log implementation
It is possible to output logs using the ADAM_DEBUG_PRINT() function defined in AdamDebug.h. The log is output to pf_<installation ID>< processId>.log in log/adam in mnt/log/pflog.tgz in the log data syslog.tgz that can be downloaded from the camera's web screen.
During development, you can also output log as UDP packet and check the log in real time using the tool.
Use of multiple AI models and multiple apps
It is possible to include multiple AI models in one app.
If you want to divide the app into multiple apps and communicate between them, please refer to the sample apps iac_server_sample_app and iac_client_sample_app.
Keep-alive monitoring
The camera firmware will try communicating to app's main thread at intervals of about 20 seconds. Please do not perform waiting processes or time-consuming processes in the main thread.
If the app does not respond for two consecutive times, it is judged that the app status is abnormal and the firmware will stop the app automatically. If the app is still within its scheduled operation period, it will automatically start again.
In addition, when the external device send AdamCGI (WEB API) to camera and the application does not return a response to the firmware, the application will stop and restart in the same way.
For example, even if the application checks the received parameters and determines that client request is not correct, app must implement a response to the firmware using ADAM_ServerResponse_Send or ADAM_ServerResponse_SendAsIs.
If you want to add another thread in your app to the keep-alive monitoring list, use ADAM_KeepAlive_Add.
CV2x and CV52 camera app compatibility and how to distinguish SoC
Camera apps for SoCs with Cv2x and CV52 are not compatible. If you have already developed a Cv2x camera app and would like to develop a CV52 camera app, you will need to convert the AI model (https://dev-partner-en.i-pro.com/space/TPFAQEN/1186890263/How+to+create+an+AI+model+for+ambaCV5X) and change the build options (https://dev-partner-en.i-pro.com/space/TPFAQEN/1158545409/SDK+installation+(v1.80+or+later)).
In the source code, following define can be used. Either one will be enabled depending on the build options.
#ifdef ADAM_TARGET_PF_ipro_ambaCV2X_linux
#ifdef ADAM_TARGET_PF_ipro_ambaCV5X_linux
This is implemented in the sample applications web_template_app, yuv_dnn_app, yuv_pose_ap, yuv_ssd_app, yuv_yolo_app, yuv_yolov5_app, and yuv_yolov8_app included in SDK v2.01 or later.
In addition, it is also possible to get SoC information using the SDK ADAM_GetSocType, which was added in firmware versions compatible with SDK 2.01 and later.
About development in Python
Python environment of i-PRO camera
The Python environment pre-installed to the i-PRO camera is shown below.
Interpreter : Python 3.7.9
External library : python adam module for ADAM API
Structure of the Python version AdamApp
The Python version AdamApp runs 2 threads, Main thread and Python thread as shown below.
Python scripts are usually executed on Python thread. On the other hand, callback functions such as stopCallback, httpCallback and appPrefCallback are executed on Main thread.
Main thread is created as soon as the Python version AdamApp commences. Then Python thread is created after Main thread completes initializing AdamApp. Python thread then launches Python interpreter, loads pymain.py, and executes Python scripts. Python thread will be automatically terminated once the Python script ends. This is followed by the termination of the Main thread and then AdamApp.