How to run multimedia examples on Duo256m running v2 firmware?

Duo
1

Hi,

I’m trying to run OpenCV Mobile example, as well as examples from duo-multimedia-examples repo on Duo256m running v2 firmware.

While they are compiled without issue, when run on device, they fail due to missing library and shared library incompatibility

Here is output from opencv_mobile_test

opencv-mobile MIPI CSI camera with cvi
Error loading shared library /mnt/system/lib/libvpu.so: No such file or directory
Error loading shared library /mnt/system/lib/libvpu.so: No such file or directory
opencv-mobile HW JPG decoder with cvi
Error loading shared library /mnt/system/lib/libvpu.so: No such file or directory
cannot find v4l device with VIDEO_CAPTURE and STREAMING
320 x 240
Trace/breakpoint trap

lipvpu seems not present in buildroot_v2 sdk at all.

Other shared libraries from /mnt/system/usr/lib/ seems to be incompatible with versions used to build examples. Here is output from video_recorder sample

Error loading shared library libvpu.so: No such file or directory (needed by ./videoRecorder)
Error relocating /mnt/system/usr/lib/libsample.so: CVI_VI_SetDevBindPipe: symbol not found
Error relocating /mnt/system/usr/lib/libsample.so: CVI_RGN_GetCanvasInfo: symbol not found
Error relocating /mnt/system/usr/lib/libsample.so: CVI_RGN_Create: symbol not found
Error relocating /mnt/system/usr/lib/libsample.so: CVI_VO_EnableChn: symbol not found
Error relocating /mnt/system/usr/lib/libsample.so: CVI_RGN_Destroy: symbol not found
Error relocating /mnt/system/usr/lib/libsample.so: CVI_VO_SetDisplayBufLen: symbol not found

Any suggestions how to fix this, especially how to get missing libvpu.so for v2 firmware?

Thank you in advance

1 Like
2

Ok, let me try to answer myself.

So, according to the sources in cvi_mpi in V2 SDK, the camera stuff has been somewhat refactored. Like, there used to be a libvpu lib, but now it’s split into libvpss and libvi.

But, unfortunately, OpenCV-Mobile (even the latest version) doesn’t know about this change. So, when you try to run a sample that’s been built with a precompiled OpenCV-Mobile, it fails.

I made a patch to fix that and recompiled OpenCV-Mobile. Then I ran the image stitching example and it worked!

1 Like
3

is it riscv or arm64 os of v2 that you are running?

1 Like
4

Hi,

I use riscv firmware

1 Like
5

Hi, I’m having the same problem. I’ve switched from SDK V1 to V2 and now I’m having issues with the camera. Could you share the patch you applied or the recompiled opencv-mobile library?

1 Like
6

Hi,

I will do during the weekend

1 Like
7

Ok, @jossvall please find attached my patches for opencv mobile. Here’s how to apply it.

Disclaimer:
OpenCV Mobile is work of GitHub - nihui/opencv-mobile: The minimal opencv for Android, iOS, ARM Linux, Windows, Linux, MacOS, HarmonyOS, WebAssembly, watchOS, tvOS, visionOS

Remarks

  • You will need the the same host tools as Buildroot SDK V2 uses.
  • I strongly advise to build in the same docker container as Buildroot SDK V2
  • I have prepared a build script - build.sh. if you going to use it, you must edit it to match your paths, especially path to the host tools directory - RISCV_ROOT_PATH variable. This is a quick hack of opencv-mobile git workflow for local build.
  • This script builds opencv-mobile 4.10. Current version is 4.11, but the patch is applicable to it as well. Change the version number in build script, if you wish.
  • The script downloads opencv source each time, but this is needed only once. Comment out the download part to save time if you going to do several builds.
  • I was trying to maintain compatibility with V1 SDK, so you can remove -DWITH_DUO_SDK_V2=ON from build script to complile original version

Then

git clone https://github.com/nihui/opencv-mobile

Replace corresponding files from archive, start and go inside container and run build.sh or follow instruction on opencv mobile github. Hope, after some time you will get compiled libs in install directory

There is one pending issue - I recommend to comment out lines 3483 to 3526 in capture_cvi.cpp. This code loads pqbin data and so far it leads to segfaults under V2 SDK. You will have lower quality of photo, but still be able to use the opencv library.

Good luck :crossed_fingers:

1 Like
8

Hi Zana

Thank you very much for taking your time and making a guide on how to apply the patch

However, these days has been something frustrating, I have followed the steps you published, also the steps that Nihui public for the compilation of Opencv Mobile, worse I have not had success, I only get a hundreds of errors of “_riscv” is not declared.

I am using the SDK v2 Docker container and the host-tools, but there are many references that are not declared, do you know what can be doing wrong?

1 Like
9

I still believe this is just a wrong paths issue. I have the following folder structure on my build host:

~
|__MilkV
        |__duo-buildroot-sdk-v2
        |  |__host-tools
        |__projects
           |__opencv-mobile

In host-tools are, umm, host tools, In opencv-mobile - clone of opencv-mobile git repo. Then, being in MilkV folder

 docker run --privileged -itd --name ocmbuild -v "$(pwd)":/home/work milkvtech/milkv-duo:latest /bin/bash
docker attach ocmbuild
cd /home/work/projects/opencv-mobile
./build.sh

Once again, check how you define RISCV_ROOT_PATH. It must contain absolute path to subfolder of host-tools, which contains riscv musl toolchain inside container, if you do docker build . In example above - /home/work/duo-buildroot-sdk-v2/host-tools/gcc/riscv64-linux-musl-x86_64

Hope, this will help

1 Like