cedrus on h616, 6.11.3 kernel, vainfo reports empty list of "Supported profile and entrypoints"

[warpme]

Hi,
This is very interesting project!
I want to give it try on mine appliance (MiniMyth2)
With 6.11.3 kernel i have well working v4l2_request on series of socs (h6/h313/h616/h618/rk3328/rk3399/rk3566/rk3588)
I'm testing current master of libva-v4l2 on h618 tvbox which perfectly plays hw accelerated h264, hevc, etc on mythtv and kodi
basic logs about what box provides (v4l2_request, drm, etc): http://termbin.com/kdx2

running vainfo (with export LIBVA_DRIVER_NAME=v4l2) with current master of libva-v4l2 gives me:

root@myth-frontend-02001a758ab4:~ # vainfo
Trying display: x11
error: can't connect to X server!
Trying display: drm
libva info: VA-API version 1.22.0
libva info: User environment variable requested driver 'v4l2'
libva info: Trying to open /usr/lib/v4l2_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: v4l2
vainfo: Supported profile and entrypoints
root@myth-frontend-02001a758ab4:~ #

maybe you have some hints why va reports no any profiles nor entrypoints?

note: minimyth2 has no systemd. For udev i'm using eudev 3.2.14 which offers udev251 api.
libva-v4l2 requires udev 252 api.
can udev 251 api level be root cause of empty list of "Supported profile and entrypoints"?

[mxsrc]

Hi, thanks for your interest in the project!

The version requirement for libudev is a conservative estimate, it's the version I worked with, as far as I can tell it should work with lower versions as well. You can circumvent this by using LIBVA_V4L2_{VIDEO,MEDIA}_PATH to explicitly set the devices to be used instead of relying on auto-lookup. The correct devices can be determined using v4l2-ctl and media-ctl respectively which should be shipped as part of v4l2-utils.

If forcing the device doesn't solve this the root issue is a different one, but maybe it's the simpler one :)

Edit: I just checked and I think I've found the reason. Currently, libva-v4l2 only supports the multi-planar M2M version, cedrus appears to use the single-planar variant. I dropped support for this while refactoring libva-v4l2-request since I had no way to validate relevant changes without the corresponding hardware. I'll try and prepare a change set to remedy this, but I cannot test it on my own.

[warpme]

Indeed - I already tried to hint driver with video/media nodes:

BusyBox v1.36.1 (2024-10-01 23:21:36 CEST) built-in shell (ash)

root@myth-frontend-02001a758ab4:~ # ls -1 /dev/media*
/dev/media0
root@myth-frontend-02001a758ab4:~ # ls -1 /dev/video*
/dev/video0
root@myth-frontend-02001a758ab4:~ # export LIBVA_V4L2_VIDEO_PATH=/dev/video0 LIBVA_V4L2_MEDIA_PATH=/dev/media0
root@myth-frontend-02001a758ab4:~ # export LIBVA_DRIVER_NAME=v4l2
root@myth-frontend-02001a758ab4:~ # vainfo
Trying display: x11
error: can't connect to X server!
Trying display: drm
libva info: VA-API version 1.22.0
libva info: User environment variable requested driver 'v4l2'
libva info: Trying to open /usr/lib/v4l2_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: Overriding V4L2 device with /dev/video0 & /dev/media0.
terminate called after throwing an instance of 'std::system_error'
  what():  Invalid argument
Aborted
root@myth-frontend-02001a758ab4:~ #

v4l2-ctl:

 v4l2-ctl -d /dev/video0  -D --list-formats-ext --list-formats-out-ext
Driver Info:
        Driver name      : cedrus
        Card type        : cedrus
        Bus info         : platform:cedrus
        Driver version   : 6.11.3
        Capabilities     : 0x84208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : cedrus
        Model            : cedrus
        Serial           :
        Bus info         : platform:cedrus
        Media version    : 6.11.3
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.11.3
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : cedrus-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'cedrus-proc' (Video Decoder): Data, Enabled, Immutable
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: AS01 (10-bit YUV 4:2:0 (AFBC 16x16), compressed)
        [1]: AS12 (YUV 4:2:0 (AFBC 16x16), compressed)
        [2]: NV12 (Y/UV 4:2:0)
        [3]: ST12 (Y/UV 4:2:0 (32x32 Linear))
ioctl: VIDIOC_ENUM_FMT
        Type: Video Output

        [0]: MG2S (MPEG-2 Parsed Slice Data, compressed)
        [1]: S264 (H.264 Parsed Slice Data, compressed)
        [2]: S265 (HEVC Parsed Slice Data, compressed)
        [3]: VP8F (VP8 Frame, compressed)
root@myth-frontend-02001a758ab4:~ #

[mxsrc]

Thanks for the input! I created a PR (#2) that should provide support for the single-planar V4L2 variant as well. It should at least make the library pick up on your device and expose the supported entrypoints (MPEG2, H264, and VP8. HEVC is not supported right now since I couldn't test that).

Your vainfo call when using the environment variables indicates another problem though, exceptions should only be used internally and not be propagated to the user. Could you run vainfo in a debugger and post the backtrace to show where the error is thrown?

[warpme]

Thx!
With single planar fix i'm getting:

vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointVLD

mythtv started report:

2024-10-16 09:11:15.968025 I  VideoProfile: Resetting decoder/render support
2024-10-16 09:11:15.974081 I  VAAPIDec: Supported/available VAAPI decoders:
2024-10-16 09:11:15.975855 I  VAAPIDec: MPEG2 Simple (Max size: 2048x2048)
2024-10-16 09:11:15.975871 I  VAAPIDec: MPEG2 Main (Max size: 2048x2048)
2024-10-16 09:11:15.975887 I  VAAPIDec: H264 Main (Max size: 2048x2048)
2024-10-16 09:11:15.975900 I  VAAPIDec: H264 High (Max size: 2048x2048)
2024-10-16 09:11:15.975912 I  VAAPIDec: H264 Constrained (Max size: 2048x2048)
2024-10-16 09:11:15.975924 I  VAAPIDec: VP8 (Max size: 2048x2048)
2024-10-16 09:11:15.977637 I  NVDEC: No NVDEC decoders found
2024-10-16 09:11:15.980400 I  V4L2_M2M: Supported/available V4L2 decoders:
2024-10-16 09:11:15.980409 I  V4L2_M2M: MPEG2(Request)
2024-10-16 09:11:15.980415 I  V4L2_M2M: H264(Request)
2024-10-16 09:11:15.980422 I  V4L2_M2M: VP8(Request)
2024-10-16 09:11:15.980430 I  V4L2_M2M: HEVC(Request)
2024-10-16 09:11:15.981374 I  DRMPRIMECtx: DRM PRIME codecs supported: None
2024-10-16 09:11:15.981565 I  VideoProfile: Available GPU interops: GL-VAAPI-DRM,GL-NVDEC,GL-DRM-PRIME
2024-10-16 09:11:15.981580 I  VideoProfile: Decoder/render support: ffmpeg      null opengl opengl-yv12
2024-10-16 09:11:15.981592 I  VideoProfile: Decoder/render support: vaapi       opengl-hw
2024-10-16 09:11:15.981601 I  VideoProfile: Decoder/render support: vaapi-dec   null opengl opengl-yv12
2024-10-16 09:11:15.981610 I  VideoProfile: Decoder/render support: v4l2        opengl-hw
2024-10-16 09:11:15.981621 I  VideoProfile: Decoder/render support: v4l2-dec    null opengl opengl-yv12 

unfortunately starting kodi gives me:

Kodi cmd.line:/usr/lib/kodi/kodi-gbm
libva info: VA-API version 1.22.0
libva error: vaGetDriverNames() failed with operation failed
libva info: User environment variable requested driver 'v4l2'
libva info: Trying to open /usr/lib/v4l2_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
/home/piotro/minimyth2-aarch64-next/images/build/usr/lib/gcc/aarch64-minimyth-linux-gnu/14.2.0/include/c++/optional:482: constexpr const _Tp& std::_Optional_base_impl<_Tp, _Dp>::_M_get() const [with _Tp = std::reference_wrapper<const V4L2M2MDevice::Buffer>; _Dp = std::_Optional_base<std::reference_wrapper<const V4L2M2MDevice::Buffer>, true, true>]: Assertion 'this->_M_is_engaged()' failed.
Aborted
root@myth-frontend-02001a758ab4:~ #

(removing export LIBVA_DRIVER_NAME=v4l2 to effectively disable vaapi - brings kodi back to working)

starting mythtv is ok, but starting playback gives following trap:
https://gist.github.com/warpme/1b8ee270469e63d1c04b2532ff59fa74

pls let me know is anything more needed to move forward with this issue :)

[mxsrc]

Okay, I will have to look into this in more detail. The error that occurs when starting Kodi seems similar to one I encountered while looking into stateful decoders. In that case, the VA-API client requests more buffers/surfaces, than V4L2 can offer. At the moment, there's a 1:1 correspondence between the two, I am looking into decoupling this but I'd like to confirm this is actually the cause here. The error mythtv gives you seems to be a different one. Can you compile a debug build of libva-v4l2 (meson setup --buildtype=debug build) to give some more details on the backtrace?

In any case, I'm afraid this will require some debugging. I am trying to acquire an SBC that uses an Allwinner SoC so I can look into this more directly, that may take a few days though.

[mxsrc]

I have fixed a few issues and believe I have uncovered the problem with H264 decoding, cedrus uses slice-based decoding and the existing code only works frame-based. Could you give #3 a try?

[warpme]

Sure :)
I compiled https://github.com/mxsrc/libva-v4l2/tree/h264-slice-based with debug profile.
Unfortunately starting kodi gives me:

Kodi cmd.line:/usr/lib/kodi/kodi-gbm
libva info: VA-API version 1.22.0
libva error: vaGetDriverNames() failed with operation failed
libva info: User environment variable requested driver 'v4l2'
libva info: Trying to open /usr/lib/v4l2_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
/home/piotro/minimyth2-aarch64-next/images/build/usr/lib/gcc/aarch64-minimyth-linux-gnu/14.2.0/include/c++/optional:482: constexpr const _Tp& std::_Optional_base_impl<_Tp, _Dp>::_M_get() const [with _Tp = std::reference_wrapper<const V4L2M2MDevice::Buffer>; _Dp = std::_Optional_base<std::reference_wrapper<const V4L2M2MDevice::Buffer>, true, true>]: Assertion 'this->_M_is_engaged()' failed.
Aborted
root@myth-frontend-02035e98a1bb:~ #

mythtv playback gives segfault

trace is:https://gist.github.com/warpme/27fd33480fa0f67d76646b1585fee01b

ps: i compiled with debug symbols only libva-v4l2 as mythtv with debug symbols needs something like 5G ram to catch segfault under gdb.
System under test was allwinner h6 with 2G.
If you need gdb trace also from mythtv - i can recompile and run test on 8G ram rk3588 board (3588 also has h264 v4l2_request working via on-chip hantro g1 decoder). Pls just let me know...

[mxsrc]

As the original issue seems solved I've merged the two other PRs. As for MythTV failing, after looking into the code I believe to understand what's causing it.

When creating the libVA driver context, MythTV seems to have trust issues with the advertised memory formats and tries to test them by creating a surface and exporting a handle to it. Unfortunately, with libva-v4l2 that mistrust is justified: The V4L2-M2M buffers corresponding to the libVA surfaces are only created after the decode context, which communicates the exact surface format. So when MythTV is probing the surfaces, the backing buffer doesn't exist yet and hence it fails.

I'll try to work around this by spoofing a buffer outside of V4L2. This might be a bit tricky though and I'll probably have to look into a local MythTV instance for testing purposes.

[warpme]

Pls let me know if there is anything i can provide to help...
(i.e. Archlinux build of MythTV frontend for your board for MiniArch Archlinux ARM)

[mxsrc]

Actually that would be really helpful. I've noticed you already have images for ROCK4s which might already work for me. The issue seems to be MythTV specific, so using that should work just fine as well. I just have the 4+, the images appear to be for 4SE and 4B only, do you reckon either would work on my board?

[warpme]

Ok - prepared small how-to about installing MythTV on Miniarch: https://github.com/warpme/miniarch/wiki/Installing-and-Setup-MythTV-on-MiniArch

Both mythtv arch package and miniarch kernel have full support of v4l2_request for mpeg2/vp8/vp9/h264/hevc.
Kernel ABI is also "conformant" with Kodi (assuming ffmpeg used by Kodi has v4l2_requrest & v4l2_drmprime patches applied) (you may look on https://github.com/warpme/minimyth2/tree/master/script/lib ffmpeg dirs. Thise ffmpegs are working well with Kodi I have in https://github.com/warpme/minimyth2/tree/master/script/mediaplayers/kodi)

Currently miniarch don't offers dedicated image for rock4+ (as I don't have this board to test) - but you may 4b or 4se. I think they are quite close :-)

Good luck!

[mxsrc]

I didn't find the time recently to look into this, but I have now set up myth TV on my Rock 4. The instructions were really helpful, the whole thing went (mostly) quite smoothly. I didn't yet check the issue itself, but now that the system is running I'll look into that soon.

[aaronhtelevic]

Hi,
I have a similar problem with coda960 on imx6.

• The VPU is successfully detected with v4l2-ctl.
• vainfo gives some output, but also doesn't list the formats

Any clue what I can do?

Goal: I want to run qtwebengine with video decoding

root@imx6:~# cat /sys/class/video4linux/video3/name
coda-video-decoder
root@imx6:~# v4l2-ctl -d /dev/video3  -D --list-formats-ext --list-formats-out-ext
Driver Info:
        Driver name      : coda
        Card type        : CODA960
        Bus info         : platform:coda
        Driver version   : 6.1.92
        Capabilities     : 0x84208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'NV12' (Y/UV 4:2:0)
        [1]: 'YU12' (Planar YUV 4:2:0)
        [2]: 'YV12' (Planar YVU 4:2:0)
        [3]: 'YUYV' (YUYV 4:2:2)
ioctl: VIDIOC_ENUM_FMT
        Type: Video Output

        [0]: 'H264' (H.264, compressed)
        [1]: 'MPG2' (MPEG-2 ES, compressed)
        [2]: 'MPG4' (MPEG-4 Part 2 ES, compressed)
root@imx6:~# vainfo
Trying display: wayland
Trying display: drm
libva info: VA-API version 1.20.0
libva info: User environment variable requested driver 'v4l2'
libva info: Trying to open /usr/lib/dri/v4l2_drv_video.so
libva info: Found init function __vaDriverInit_1_20
libva info: Overriding V4L2 device with /dev/video3 & .
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.20 (libva 2.20.1)
vainfo: Driver version: v4l2
vainfo: Supported profile and entrypoints