在 v4l2 中设置视频格式以使用 gstreamer 进行流式传输

Question

我想使用 gstreamer (gst-launch-1.0) 通过 UDP 从连接到 raspberry pi (CM4) 的摄像头将视频信号流式传输到远程客户端。即使在我使用 v4l2 将格式设置为 MJPG 之后，我使用的 gstreamer 管道总是恢复为未压缩的 YUYV 像素格式。

这是我的管道：

pi@cm4:~ $ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=MJPG
pi@cm4:~ $ gst-launch-1.0 -v v4l2src device=/dev/video0  ! "video/x-raw, width=1920, height=1080, pixelformat=MJPG" ! rndbuffersize max=65000 ! udpsink host=127.0.0.1 port=1234
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstRndBufferSize:rndbuffersize0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstRndBufferSize:rndbuffersize0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, pixelformat=(string)MJPG, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive

尽管管道似乎接受“pixelformat=(string)MJPG”，但格式为 YUY2，最大帧率为 5fps。如果我将帧率设置为高于 5/1 的值，它会失败并显示：

** (gst-launch-1.0:16205): CRITICAL **: 21:36:05.076: gst_adapter_take_buffer: assertion 'GST_IS_ADAPTER (adapter)' failed

** (gst-launch-1.0:16205): CRITICAL **: 21:36:05.076: gst_adapter_available: assertion 'GST_IS_ADAPTER (adapter)' failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

执行 gstreamer 管道后，v4l2-ctl 确认视频格式恢复为 YUYV。

如何强制 gstreamer 管道使用 MJPG 1920x1080 并启用更高的帧速率？

相机是佳能 5D iv，可产生清晰的 HDMI 输出，最高可达 60fps 的全高清。相机 HDMI 输出连接到支持 1920x1080 60fps 的 HDMI 转 USB 视频捕获 (mirabox)。视频采集盒通过USB3-PCIe适配器连接CM4

这是支持的格式列表：

pi@cm4:~ $ v4l2-ctl -d 0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'MJPG' (Motion-JPEG, compressed)
            Size: Discrete 1920x1080
                    Interval: Discrete 0.017s (60.000 fps)
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.040s (25.000 fps)
                    Interval: Discrete 0.050s (20.000 fps)
                    Interval: Discrete 0.100s (10.000 fps)
            [..... deleted lower resolution formats...]
    [1]: 'YUYV' (YUYV 4:2:2)
            Size: Discrete 1920x1080
                    Interval: Discrete 0.200s (5.000 fps)
            Size: Discrete 1600x1200
                    Interval: Discrete 0.200s (5.000 fps)
            [..... deleted lower resolution formats...]

Answer 1

这里设置像素格式其实是不正确的。 MJPEG 不是“原始”视频的像素格式。

尝试

v4l2src device=/dev/video0  ! image/jpeg, width=1920, height=1080, framerate=30/1 ! ..

请注意，相机会 return 为您提供 jpeg 图像数据，因此如果要显示图像，您需要一个 jpeg 解码器。

在 v4l2 中设置视频格式以使用 gstreamer 进行流式传输

setting the video format in v4l2 for streaming with gstreamer

gstreamer

v4l2

gst-launch-1.0