USB UVC support for device_next

[josuah]

I would be nice to have UVC support for Zephyr USB stack.

• usb: device_next: new USB Video Class (UVC) implementation #76798

Also I need it soon, so I will gather ideas and try get a PoC going.
Nothing presentable yet, only enough to get video frames in/out as an early goal.

There seems to be an existing effort merged into Zephyr, which lead to a Zephyr sample:
#17194 - Video 4 Zephyr - Video API, Drivers and Samples

[josuah]

I am almost ready with this: now preparing the final integration with the Video API.

The descriptors can be encoded on the devicetree like this:

&udc0 {
        /* IN and OUT counting together as one endpoint */
        num-bidir-endpoints = <2>;

        /* full-speed: 12 Mhz, high-speed: 480 Mhz, super-speed: 5000 Mhz */
        maximum-speed = "super-speed";

        uvc0: zephyr_uvc0 {
                compatible = "zephyr,uvc";
                status = "okay";

                source = <&video_app>;

                port {
                        uvc0_in: endpoint {
                                remote-endpoint = <&app_out>;
                        };
                };

                format_mjpeg {
                        compatible = "zephyr,uvc-format-mjpeg", "zephyr,uvc-format";
                        fourcc = "JPEG";

                        640x400 {
                                size = <640 400>;
                                fps = <15>;
                        };

                        64x64 {
                                size = <64 64>;
                                fps = <15>;
                        };
                };

                format_uncompressed {
                        compatible = "zephyr,uvc-format-uncompressed", "zephyr,uvc-format";
                        fourcc = "YUYV";
                        guid = [UVC_GUID_YUY2];
                        bits-per-pixel = <16>;

                        640x400 {
                                size = <640 400>;
                                fps = <15>;
                        };

                        64x64 {
                                size = <64 64>;
                                fps = <30>;
                        };
                };
        };
};

This gets exposed to the host as USB descriptors, and to Zephyr as struct video_caps.

I will make a draft pull request with a sample for any device_next supported device, and the sw_generator.c as source.

[uLipe]

@josuah do you have the draft branch where I can try out with my Nicla Visio?

[josuah]

Here it is! #76798
I am currently bringing-up the UART to get logs instead of default USB: I will need to log the transfers to make sure I did not misunderstand the USB API.

[ehughes]

@ehughes I had some issue with NXP LinkServer to get the image on the MCXN947, but will try again once I figure out what went wrong. I can also try with JLink also supported. SWD alone does not permit flashing... Maybe without XIP though it'll work.

I had it enumerate with the STM32H743 board from WeAct Studio though. It is recognized as an /dev/video2 interface (assuming your laptop webcam is /dev/video0). So likely it is not too far away from being mergeable.

FYI:

I use segger/jlink on MCXN947 and it works well. I typically use segger ozone to debug as you can just point it towards /build/zephyr/zephyr.elf. It detects when a rebuild happens and can auto reload.

[josuah]

Hello @ehughes and thank you for your advises.
I ended-up making OpenOCD working, but had a crash of the UDC driver. I will come back to it soon.

[josuah]

From GDB I observe an Usage Fault on ehciState->registerBase->USBCMD |= USBHS_USBCMD_RST_MASK; (upon writing the value back, maybe something happening from IRQ context...) but it works well on the CDC example... still no UVC on the FRDM MCXN947... sorry! Will try again next time I have a chance.