Gstreamer UDP 流从 Tello 无人机到 RTP
Gstreamer UDP stream from Tello drone into RTP
我正在尝试使用 Gstreamer 将视频从 Tello 无人机流式传输到 RTP，以便进一步将其用于 jetson 推理。接收 UDP 包的计算机是 Jetson Nano。迄今为止最成功的命令是
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)960, height=(int)720, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! videoconvert ! autovideosink sync=false
当 运行 它在 shell 时，带有视频的大 window 开始播放。现在我想将此视频转发到 RTP。我尝试了 x264enc、rtph264depay、rtph264pay 的各种组合，但每次管道因内部数据流错误而中断。上次尝试的结果：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! videoconvert ! x264enc ! rtph264pay ! udpsink
输出：
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)960, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad1: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)960, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)24/1
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.971564245
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
如果有人已经这样做了，请分享 gstreamer 的正确行。谢谢
你的问题是decodebin选择了输出到NVMM内存的nvv4l2decoder。 videoconvert 无法从 NVMM 内存读取。您可以改用 nvvidconv，它可以从 NVMM 读取并输出到系统内存中。
但是，重新编码为h264并不一定要解码h264。
这个简单的管道应该可以完成这项工作：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! rtph264pay config-interval=1 ! udpsink host=X.X.X.X port=YYYY auto-multicast=0
其中 X.X.X.X 是接收者的地址（例如本地主机为 127.0.0.1），YYYY 是端口，例如 5000。
对于使用多播（不要与 wifi 一起使用）：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! rtph264pay config-interval=1 ! udpsink host=host=224.1.1.1 port=5000
或者如果你想用硬件解码re-encode：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! nvv4l2decoder ! nvv4l2h264enc insert-sps-pps=1 ! rtph264pay ! udpsink host=X.X.X.X port=YYYY auto-multicast=0
如果您想使用原来的管道：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! nvvidconv ! x264enc insert-vui=1 ! rtph264pay ! udpsink host=X.X.X.X port=YYYY auto-multicast=0
我正在尝试使用 Gstreamer 将视频从 Tello 无人机流式传输到 RTP，以便进一步将其用于 jetson 推理。接收 UDP 包的计算机是 Jetson Nano。迄今为止最成功的命令是
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)960, height=(int)720, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! videoconvert ! autovideosink sync=false
当 运行 它在 shell 时，带有视频的大 window 开始播放。现在我想将此视频转发到 RTP。我尝试了 x264enc、rtph264depay、rtph264pay 的各种组合，但每次管道因内部数据流错误而中断。上次尝试的结果：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! videoconvert ! x264enc ! rtph264pay ! udpsink
输出：
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=(int)2, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)960, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)24/1
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad1: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)960, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)24/1
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.971564245
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
如果有人已经这样做了，请分享 gstreamer 的正确行。谢谢
你的问题是decodebin选择了输出到NVMM内存的nvv4l2decoder。 videoconvert 无法从 NVMM 内存读取。您可以改用 nvvidconv，它可以从 NVMM 读取并输出到系统内存中。
但是，重新编码为h264并不一定要解码h264。 这个简单的管道应该可以完成这项工作：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! rtph264pay config-interval=1 ! udpsink host=X.X.X.X port=YYYY auto-multicast=0
其中 X.X.X.X 是接收者的地址（例如本地主机为 127.0.0.1），YYYY 是端口，例如 5000。
对于使用多播（不要与 wifi 一起使用）：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! rtph264pay config-interval=1 ! udpsink host=host=224.1.1.1 port=5000
或者如果你想用硬件解码re-encode：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! h264parse ! nvv4l2decoder ! nvv4l2h264enc insert-sps-pps=1 ! rtph264pay ! udpsink host=X.X.X.X port=YYYY auto-multicast=0
如果您想使用原来的管道：
gst-launch-1.0 -v udpsrc port=11111 caps="video/x-h264, stream-format=(string)byte-stream, width=(int)300, height=(int)300, framerate=(fraction)24/1, skip-first-bytes=2" ! queue ! decodebin ! nvvidconv ! x264enc insert-vui=1 ! rtph264pay ! udpsink host=X.X.X.X port=YYYY auto-multicast=0