How to modify buildroot for IMX219/IMX327/IMX462 support on Duo S?

markwu2001 · July 29, 2024, 11:40pm

Hello everyone, thank you for all the support so far!

I am trying to take 400ms exposure images using an IMX327 sensor on J2 (15 pin MIPI connector) of the milk v duo s. I’ve tried a few things and had no luck so I’m hoping somebody who knows how to interface with the ISP better is able to pitch in with some ideas to support this sensor.

I see 77 files which reference the IMX327 in the buildroot codebase which suggests that there is native support for this sensor.

What I’ve attempted so far:

Modify the defconfig file to include support for 3 IMX327 configs

CONFIG_SENSOR_SONY_IMX327=y
CONFIG_SENSOR_SONY_IMX327_2L=y
CONFIG_SENSOR_SONY_IMX327_SLAVE=y

2. Build Image successfully and flash to milk v duo s

Build board milkv-duos-emmc success!
Create eMMC image successful: out/milkv-duos-emmc-20240730-0501.zip

Duplicate officially provided sensor_cfg_OV5647_J2.ini, unlink and link sensor_cfg.ini to the new file.

[root@milkv-duo]/mnt/data# cp sensor_cfg_OV5647_J2.ini IMX327_cfg.ini
[root@milkv-duo]/mnt/data# unlink sensor_cfg.ini
[root@milkv-duo]/mnt/data# ln IMX327_cfg.ini sensor_cfg.ini
[root@milkv-duo]/mnt/data# vi sensor_cfg.ini

Modify sensor_cfg.ini to change name of sensor to SONY_IMX327_2L_MIPI_2M_30FPS_12BIT and change i2c address to 1a. I’m assuming everything else should remain the same because the camera module and J2 are both compatible with the raspberry pi camera standard.

;section for source
[source]
;type = SOURCE_USER_FE
dev_num = 1
;section for sensor
[sensor]
;sensor name
name = SONY_IMX327_2L_MIPI_2M_30FPS_12BIT
;bus/i2c dev number
bus_id = 2
sns_i2c_addr = 1a
mipi_dev = 0
lane_id = 5, 3, 4, -1, -1
pn_swap = 0, 0, 0,  0,  0

Run sensor_test. This is what I get before it crashes (does not show me menu for data dump)

[sys_vi_init]-41: MMF Version:7e0cc6a08-musl_riscv64
[SAMPLE_COMM_SNS_ParseIni]-1950: Parse /mnt/data/sensor_cfg.ini
[parse_source_devnum]-1605: devNum =  1
[parse_sensor_name]-1686: sensor =  SONY_IMX327_2L_MIPI_2M_30FPS_12BIT
[parse_sensor_busid]-1714: bus_id =  2
[parse_sensor_i2caddr]-1725: sns_i2c_addr =  1a
[parse_sensor_mipidev]-1736: mipi_dev =  0
[parse_sensor_laneid]-1747: Lane_id =  5, 3, 4, -1, -1
[parse_sensor_pnswap]-1758: pn_swap =  0, 0, 0,  0,  0
[SAMPLE_PLAT_SYS_INIT]-72: common pool[0] BlkSize 3133440
ISP Vipipe(0) Allocate pa(0x96c30000) va(0x0x3fcd9f4000) size(291120)
stSnsrMode.u16Width 1920 stSnsrMode.u16Height 1080 25.000000 wdrMode 0 pstSnsObj 0xf29a8
[SAMPLE_COMM_VI_StartMIPI]-483: sensor 0 stDevAttr.devno 0

run camera-test.sh for more verbose error.

[SAMPLE_COMM_SNS_ParseIni]-1950: Parse /mnt/data/sensor_cfg.ini
[parse_source_devnum]-1605: devNum =  1
[parse_sensor_name]-1686: sensor =  SONY_IMX327_2L_MIPI_2M_30FPS_12BIT
[parse_sensor_busid]-1714: bus_id =  2
[parse_sensor_i2caddr]-1725: sns_i2c_addr =  1a
[parse_sensor_mipidev]-1736: mipi_dev =  0
[parse_sensor_laneid]-1747: Lane_id =  5, 3, 4, -1, -1
[parse_sensor_pnswap]-1758: pn_swap =  0, 0, 0,  0,  0
MMF Version:7e0cc6a08-musl_riscv64
Create VBPool[0], size: (3110400 * 3) = 9331200 bytes
Create VBPool[1], size: (3110400 * 3) = 9331200 bytes
Create VBPool[2], size: (2764800 * 1) = 2764800 bytes
Total memory of VB pool: 21427200 bytes
Initialize SYS and VB
Initialize VI
ISP Vipipe(0) Allocate pa(0x96871000) va(0x0x3fbe270000) size(291120)
stSnsrMode.u16Width 1920 stSnsrMode.u16Height 1080 25.000000 wdrMode 0 pstSnsObj 0x3fbf0b4490
[SAMPLE_COMM_VI_StartMIPI]-483: sensor 0 stDevAttr.devno 0
vi init failed. s32Ret: 0xffffffff !
init middleware failed! ret=ffffffff

It seems that vi init and middleware init is failing. I am currently trying to learn how these systems work using Sophon’s documentation but I still don’t know what steps to take to support this sensor and if I should spend time double checking the hardware.

If the author of the OV5647 raspberry pi camera support is reading this, could you let me know what steps you took to add this support? Thank you very much! @Carbon

piotr-de · July 31, 2024, 2:54pm

The Duo-S has RPi compatible pinout on J2, but is it voltage-compatible? MilkV uses 1.8V logic levels as far as I know, while RPi-compatible cameras expect 3.3V levels of the RaspberryPi

piotr-de · July 31, 2024, 3:02pm

If you make it work pls let us know, I’m also trying to accomplish a simillar thing

markwu2001 · July 31, 2024, 11:34pm

Yep, it should be voltage compatible. MIPI standard is usually 1.8V so those lines should be 1.8V on the duo s as well as the sensor.

Raspberry Pi camera modules expect 3.3V I2C on the host side and will have 3.3V->1.8V level shifters to accomodate for the voltage used by the sensor. It should be the case that milk-v duo s uses 3.3V I2C on J2 considering its worked with the raspberry pi camera V1.

Last is power. Raspberry pi camera expects 3.3V which is the case on duo s J2.

Right now, I’d like to explore controlling the ISP directly with its C reference. I’ll let you know if I get anywhere. I am also considering switching to the RV1103 by rockchip because they wrote V4L2 drivers for it.

Also correction to title. I meant IMX290/IMX327/IMX462 which are part of the STARVIS family. I keep getting the sensor names confused haha.

piotr-de · August 1, 2024, 7:37am

Seems logical I also was considering switching to RV1103 (Luckfox Pico) but in general MilkV seems like a better solution, so I really hope this can be figured out and that there’s a way to make STARVIS cams work Maybe @Carbon can help?

piotr-de · August 1, 2024, 7:40am

One thing though, take a look at this thread (I don’t know chinese, so can only understand what google translate provides) milkVDUO是否能打个转接板用树莓派的摄像头 - #5 by jesson

I didn’t play with Duo S, however, it seems smaller boards (Duo, Duo256) do indeed require external level shifting. But it sounds quite convincing, that as it worked with RPi camera V1, DuoS should be hardware-wise compatible.

piotr-de · August 8, 2024, 11:51am

Bump
Maybe the v4l2 will make it easier to implement? @Carbon

paul.effect · January 25, 2025, 12:10pm

The smaller boards have only 1 camera connector. Which is intended to be used with the milkv camera. Which uses 1v8 signalling. Not applicable.

@markwu2001 Did you manage to get it up and running? Id very much like to use the 415. Frankly, I cannot understand why every project similar to the DUO chose some obscure sub-par sensor. The starvis1/ 2 family is one helluva bang for the buck.