【Milk-V Duo】CV1800 TPU YOLO 开发及测试

Duo
1

下载 tdl sdk 和 tpu sdk

首先测试 tdl-sdk

  • 进入 cvitek-tdl-sdk-cv180x/sample/cvi_yolo/compile_sample.sh
    • 增加工具链位置指定：
      export PATH=/path_to_sdk/duo-src/host-tools/gcc/riscv64-linux-musl-x86_64/bin/:$PATH
    • 找到 cvi_yolo 选中 sample_yolov5，把里面图片的分辨率改为图片的分辨率，这里是640*640
    • ./compile_sample.sh
  • 找到 cvi_yolo 选中 sample_yolov5，使用上一贴转化出来的 yolov5n_int8_fuse.cvimodel 和 bus.jpg 测试：
    shell [root@milkv-duo]~/tpu-sdk/test3# ./sample_yolov5_roi yolov5n_int8_fuse.cvimodel bus.jpg ion ioctl fail:: Out of memory reopen ion dev reopen ion dev success ion ioctl fail:: Out of memory reopen ion dev reopen ion dev success ion ioctl fail:: Out of memory reopen ion dev reopen ion dev success Assertion failed: 0 (../../src/soc/common/cvi_device_mem.cpp: mem_alloc_raw: 451) Aborted
  • 根据官方文档提到的：
    • YOLODevelopmentGuide
    • yolov5m模型的TDL_SDK导出方式onnx 在cv181x上ion分配失败，可能是规格不足，yolov5n跟yolov5m应该差不了多少。也许duo s或者256就可以，但我手里都没有，就算了。
    • 又尝试重新导出yolov5s，运行后却直接 Killed 了，可能哪里没设置对
    • 随后又尝试了一下yolov8n，在[YOLODevelopmentGuide]里面提到的frp里面有一个官方的yolov8n模型，yolov8n_int8.cvimodel，配合 cvi_yolo 选中 sample_yolov8 测试结果依然是ion分配失败

cvitek_tpu_samples

  • 事情到这里似乎陷入了一个僵局，但我又从[YOLODevelopmentGuide]里面翻到另一个sdk，叫cvitek_tpu_samples
  • 里面有个 samples_extra/detector_yolov8n_fused_preprocess
  • 根据这个例程修改了下，测试通过，仓库

运行

  • 将动态库下载到板子上
  • export LD_LIBRARY_PATH=“.”
  • ./cvi_yolov8_first yolov8n_int8.cvimodel
  • 如果显示在SPI屏幕上，大概1s一帧；
  • 如果不做显示处理，大概2s一帧
  • 由于目前SDK里面的opencv是没有video和videoio模块，所以本来准备了对焦测试员的视频想采视频流来识别的想法没有实现。

摄像头驱动与视频测试的尝试

  • 人脸识别教程
  • 根据人脸识别教程编译了 sample_vi_fd.c 但是运行始终打印 vi init failed
VI_SDK_IOC_S_CTRL - vi_sdk_disable_chn NG, Invalid argument
VI_SDK_IOC_S_CTRL - vi_sdk_destroy_pipe NG, Invalid argument
VI_SDK_IOC_S_CTRL - vi_sdk_disable_dev NG, Invalid argument
vi init failed. s32Ret: 0xffffffff !
init middleware failed! ret=ffffffff
  • 上一帖用例程测试过摄像头是可以正常驱动的 sample_vi_fd ，可能例程还是有哪里不对。
2

Dear Tinnu,
Thank you for your information.
I also saw same error message on my Duo 64M board which uses firmware of version 1.0.9.
When I used the binary file obtained from the “bin” directory on the github page, I saw an different error message as follows.
I hope someone overcome this situation.

[root@milkv-duo]~/face_detection# ./sample_vi_fd scrfd_768_432_int8_1x.cvimodel
[SAMPLE_COMM_SNS_ParseIni]-1950: Parse /mnt/data/sensor_cfg.ini
[parse_source_devnum]-1605: devNum = 1
[parse_sensor_name]-1686: sensor = GCORE_GC2083_MIPI_2M_30FPS_10BIT
[parse_sensor_busid]-1714: bus_id = 1
[parse_sensor_i2caddr]-1725: sns_i2c_addr = 37
[parse_sensor_mipidev]-1736: mipi_dev = 0
[parse_sensor_laneid]-1747: Lane_id = 3, 2, 4, -1, -1
[parse_sensor_pnswap]-1758: pn_swap = 0, 0, 0, 0, 0
MMF Version:7e0cc6a08-musl_riscv64
Create VBPool[0], size: (3110400 * 2) = 6220800 bytes
Create VBPool[1], size: (1382400 * 2) = 2764800 bytes
Create VBPool[2], size: (2764800 * 1) = 2764800 bytes
Total memory of VB pool: 11750400 bytes
Initialize SYS and VB
Initialize VI
ISP Vipipe(0) Allocate pa(0x82fa8000) va(0x0x3fd9d01000) size(291016)
stSnsrMode.u16Width 1920 stSnsrMode.u16Height 1080 25.000000 wdrMode 0 pstSnsObj 0x3fda84b878
[SAMPLE_COMM_VI_StartMIPI]-483: sensor 0 stDevAttr.devno 0
awbInit ver 6.8@2021500
0 R:1400 B:3100 CT:2850
1 R:1500 B:2500 CT:3900
2 R:2300 B:1600 CT:6500
Golden 1024 1024 1024
WB Quadratic:0
isWdr:0
ViPipe:0,===GC2083 1080P 30fps 10bit LINE Init OK!===

cvi_bin_isp message
gerritId: 36403 commitId: c69c5863e
md5: cab880835a2ad5184de5ed7762404b84
sensorNum 1
sensorName0 2083

PQBIN message
gerritId: 80171 commitId: 5c9d8fc5d
md5: ba5a510e093ad42db6788e6c2d13169e
sensorNum 3
sensorName0 2053

author: wanqiang.he desc: 思博慧CV1812H_GC2083_RGB_mode_V1.0.0
createTime: 2023-08-04 16:48:08version: V1.1
tool Version: v3.0.5.24 mode:

sensorName(0) mismatch, mwSns:2083 != pqBinSns:2053
Cannot open ‘/dev/cvi-vo’: 2, No such file or directory
[SAMPLE_COMM_ISP_Thread]-95: ISP Dev 0 running!
Initialize VPSS
---------VPSS[0]---------
Input size: (1920x1080)
Input format: (19)
VPSS physical device number: 1
Src Frame Rate: -1
Dst Frame Rate: -1
--------CHN[0]-------
Output size: (1280x720)
Depth: 1
Do normalization: 0
Src Frame Rate: -1
Dst Frame Rate: -1
----------------------
--------CHN[1]-------
Output size: (1280x720)
Depth: 1
Do normalization: 0
Src Frame Rate: -1
Dst Frame Rate: -1
----------------------

Bind VI with VPSS Grp(0), Chn(0)
Attach VBPool(0) to VPSS Grp(0) Chn(0)
Attach VBPool(1) to VPSS Grp(0) Chn(1)
Initialize VENC
venc codec: h264
venc frame size: 1280x720
Initialize RTSP
rtsp://169.254.178.29/h264
prio:0
failed! ret=0xc0010101, at …/…/sample/sample_vi_fd.c:247
destroy middleware
ISP Vipipe(0) Free pa(0x82fa8000) va(0x0x3fd9d01000)
stop VPSS (0)
[root@milkv-duo]~/face_detection#~/face_detection# ifconfig
eth0 Link encap:Ethernet HWaddr 26:65:F4:83:A2:88
inet addr:169.254.178.29 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::87e4:e8e4:84b6:4792/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:3438 (3.3 KiB)
Interrupt:22

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:320 errors:0 dropped:0 overruns:0 frame:0
TX packets:320 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:19256 (18.8 KiB) TX bytes:19256 (18.8 KiB)

usb0 Link encap:Ethernet HWaddr E2:1C:EF:C9:FC:42
inet addr:192.168.42.1 Bcast:192.168.42.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

[root@milkv-duo]~/face_detection#