h264_v4l2m2m acceleration in Raspberry Pi 4 - 64 bits

[randm-ch]

Good evening! I'm trying to get hardware acceleration to work in the Jellyfin container on a RPi4 64bit OS (Manjaro ARM), so far without luck. As OMX doesn't work on 64bit (and is getting deprecated anyway) it is suggested in different places to use V4L2. This does work with ffmpeg on the host machine without problems (with ffmpeg -i ... -c:v h264_v4l2m2m ...), but inside the container it doesn't. I have added devices /dev/video10 (as well as 11 & 12) to the container.

See also: jellyfin/jellyfin-ffmpeg#310

---

Expected Behavior

L4L2 hardware acceleration works.

Current Behavior

L4L2 hardware acceleration doesn't work. The error is:

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_v4l2m2m))
  Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0xaaaaebe39a00] Using device /dev/video11
[h264_v4l2m2m @ 0xaaaaebe39a00] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0xaaaaebe39a00] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0xaaaaebe39a00] Encoder requires yuv420p pixel format.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0xaaaaebe15020] Qavg: 65536.000
[aac @ 0xaaaaebe15020] 2 frames left in the queue on closing
Conversion failed!

When trying to run ffmpeg manually with -pix_fmt yuv420p as suggested in jellyfin/jellyfin-ffmpeg#310 I get this error:

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_v4l2m2m))
  Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0xaaaafae82a00] Using device /dev/video11
[h264_v4l2m2m @ 0xaaaafae82a00] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0xaaaafae82a00] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0xaaaafae82a00] Failed to set gop size: Invalid argument
Output #0, hls, to '/config/data/transcodes/224b83dbe3d5c4dd0430795641fcd081.m3u8':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (h264_v4l2m2m), yuv420p, 3840x2160 [SAR 1:1 DAR 16:9], q=-1--1, 37433 kb/s, 24 fps, 90k tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 h264_v4l2m2m
    Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp, 640 kb/s (default)
    Metadata:
      encoder         : Lavc58.91.100 aac
[h264_v4l2m2m @ 0xaaaafae82a00] VIDIOC_STREAMON failed on output context
Video encoding failed
[aac @ 0xaaaafae5e020] Qavg: 65536.000
[aac @ 0xaaaafae5e020] 2 frames left in the queue on closing
double free or corruption (out)
Aborted (core dumped)

Steps to Reproduce

1. Start linuxserver/jellyfin docker container on a RPi4 running a 64bit OS, using required devices
2. Configure hardware acceleration in Jellyfin settings, selecting "Exynos L4L2 MFC"
3. Play a video which needs encoding

Environment

OS: Manjaro ARM
CPU architecture: arm64
How docker service was installed: ghcr.io/linuxserver/jellyfin via docker-compose

Command used to create docker container (run/create/compose/screenshot)

services:

  jellyfin:
    image: ghcr.io/linuxserver/jellyfin
    environment:
      PUID: "${UID}"
      PGID: "${GID}"
      TZ: "${TIMEZONE}"
      JELLYFIN_PublishedServerUrl: "${HOST_NAME}:8096"
    ports:
      - "8096:8096"
    volumes:
      - "${VOLUMES_DIR}/jellyfin:/config"
      - "${STORAGE_DIR}:/data"
    devices:
      - "/dev/video10:/dev/video10"
      - "/dev/video11:/dev/video11"
      - "/dev/video12:/dev/video12"

Docker logs

https://gist.github.com/randm-ch/285b175f95dac3fd02c3de3f86728c05

[github-actions]

Thanks for opening your first issue here! Be sure to follow the bug or feature issue templates!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ananthb]

I'm having the exact same issue. Host OS is Ubuntu 64 bit, and I would love to get HW acceleration for HEVC content.

[SonnTakuu]

same problem

[da-anda]

same

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[da-anda]

still not working as far as I can tell

[RHITNL]

Same Here. Running docker on 64bits arm ubuntu server 21.04

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Pajn]

Not stale

[the4anoni]

When using v4l2 on Raspberry Pi OS ArmHF I had same error. Here Is my compose:
#130

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Sn00zEZA]

also would like this issue resolve for Rpi4 running raspios-bullseye-arm64-lite
jellyfin/jellyfin-ffmpeg#310

Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_v4l2m2m))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0x55a1718540] Using device /dev/video11
[h264_v4l2m2m @ 0x55a1718540] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x55a1718540] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x55a1718540] Encoder requires yuv420p pixel format.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a171db10] Qavg: 65536.000
[aac @ 0x55a171db10] 2 frames left in the queue on closing
Conversion failed!

/usr/lib/jellyfin-ffmpeg/ffmpeg -autorotate 0 -i file:"/data/movies/Bluray-1080p.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_v4l2m2m -pix_fmt nv21 -b:v 2552000 -maxrate 2552000 -bufsize 5104000 -level 41 -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -g:v:0 72 -keyint_min:v:0 72 -sc_threshold:v:0 0 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*dar)\,1280)/64)*64:trunc(ow/dar/2)*2,format=yuv420p" -codec:a:0 aac -ac 6 -ab 448000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/transcode/3684e786242cae71a38699487a97b21e%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/transcode/3684e786242cae71a38699487a97b21e.m3u8"

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[randm-ch]

Still unresolved

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[randm-ch]

Still unresolved. But it looks like the base image fixed the issue: jellyfin/jellyfin#7291
However, sounds like performance isn't great yet.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[zotabee]

Still unresolved

[ioogithub]

I have been struggling with this issue as well. Using the official Raspberry Pi OS 64 bit Ubuntu image Ubuntu 21.10.

Still unresolved. But it looks like the base image fixed the issue: jellyfin/jellyfin#7291 However, sounds like performance isn't great yet.

How can I test this myself, do I need to use the latest tag or do I need to pull the nightly image?

I am using the latest , installed a few days ago and I can confirm it still does now work, ffmpeg exits with status 1 when a file that needs to be transcoded is played.

Has anyone found any solution to this issue?

[randm-ch]

I am using the latest

If you use jellyfin/jellyfin:unstable – not linuxserver/jellyfin – then it should work. Haven't tested it myself tho.

[cacafuti8888]

I am using the latest

If you use jellyfin/jellyfin:unstable – not linuxserver/jellyfin – then it should work. Haven't tested it myself tho.

Yeah, he's right. Tested and working fine, getting around 1.3x speed transcoding 1080p x265 > 1080p x265. But jellyfin still has some annoying bugs to me.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[randm-ch]

Still unresolved

[j0nnymoe]

So reading through these comments, It seems like it's not a container issue, it's more a jellyfin/hardware issue?

[ioogithub]

According to the change log, the ffmpeg version has been updated:
https://docs.linuxserver.io/images/docker-jellyfin#versions

Does this mean the hardware encoding now works with version 11.06.22 of this image? Has anyone tested it?

[da-anda]

from what I can tell, h264 should be working now. At least my transcoding logs indicate that h265 (native) ->h264_v4l2m2m seems to work. The PI4 will still struggle because of currently missing HW accelerated decoding of h265, but at least the encoding part this ticket is about should be working in HW now.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[romikette]

I think my issue was similar and I solved mine running this in the container console:

usermod -aG video jellyfin

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[drizuid]

As it appears this was a jellyfin issue, not a container issue (mixed with a permissions issue on the host) I will close this as we cannot resolve upstream dev issues or host permissions issues.