Gstreamer：如何在不重新编码的情况下将 rtpvp8depay 传输到 webmmux？
Gstreamer: How to pipe rtpvp8depay into webmmux without reencoding?
我从提供浏览器的 Webrtc 收到一个 RTP 流，该流使用 janus 网关解密。仅接收视频 rtp 数据包后，将中继到本地多播组以进行测试。
所以，假设我在 udp 端口上接收到 vp8 编码的 rtp 数据包。我还可以随时请求新的关键帧。
问题管道：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! webmmux streamable=true ! filesink location=/tmp/test.webm
产生错误
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2933): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.039571113
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Kurento 项目提供了一个名为 "vp8parse" 的 gstreamer 插件，它解决了这个问题：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8parse ! webmmux streamable=true ! filesink location=/tmp/test.webm
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:sink: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
HERE THE PIPELINE BLOCKS UNTIL A KEYFRAME IS RECEIVED
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstMatroskamuxPad:video_0: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/webm
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"
看看 vp8parse source 似乎这个插件除了将 rtpvp8depay 提供的帧原封不动地传送到它的接收器之外什么也没做，但还将 src 上限设置为视频宽度、高度和帧率。
另一种可行的管道是：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8dec ! vp8enc ! webmmux streamable=true ! filesink location=/tmp/test.webm
但使用 vp8dec ！ vp8enc 显然没有多大意义，因为我已经收到了一个 vp8 编码流。
现在我的问题是如何在不重新编码流并且不依赖 vp8parse 的情况下解决这个问题？如果别无选择，我似乎必须使用它，但由于这是目前无法通过标准 gstreamer 插件包获得的插件，我想避免这种情况。是否可以将上限强制为特定的宽度、高度、帧率，这样 webmmux 就不会抱怨了？因为我认为这就是第一个管道未协商的原因。
我试过使用像 rtpvp8depay 这样的 capsfilter！ capsfilter 上限="video/x-vp8,width=640,height=480,framerate=10/1" ! webmmux 但它也不协商。
这是错误 747208，已在上游修复，
但是你系统中的 Gstreamer 版本可能是旧的
（Ubuntu 14.04 中的 1.2.4）并且仍然受到影响。
作为此类旧版本的解决方法，
如果您知道视频的帧大小
你可以在 depayloader 之后使用 capssetter 元素
手动设置 depayloader 遗漏的上限：
gst-launch-1.0 -v \
udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 \
! "application/x-rtp, payload=100, clock-rate=90000" \
! rtpvp8depay ! capsetter caps="video/x-vp8,width=640,height=480" \
! webmmux streamable=true ! filesink location=/tmp/test.webm
我从提供浏览器的 Webrtc 收到一个 RTP 流，该流使用 janus 网关解密。仅接收视频 rtp 数据包后，将中继到本地多播组以进行测试。
所以，假设我在 udp 端口上接收到 vp8 编码的 rtp 数据包。我还可以随时请求新的关键帧。
问题管道：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! webmmux streamable=true ! filesink location=/tmp/test.webm
产生错误
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2933): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.039571113
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Kurento 项目提供了一个名为 "vp8parse" 的 gstreamer 插件，它解决了这个问题：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8parse ! webmmux streamable=true ! filesink location=/tmp/test.webm
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:sink: caps = "video/x-vp8\,\ framerate\=\(fraction\)0/1"
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps = "application/x-rtp\,\ payload\=\(int\)100\,\ clock-rate\=\(int\)90000\,\ media\=\(string\)video\,\ encoding-name\=\(string\)VP8-DRAFT-IETF-01"
HERE THE PIPELINE BLOCKS UNTIL A KEYFRAME IS RECEIVED
/GstPipeline:pipeline0/KmsVp8Parse:kmsvp8parse0.GstPad:src: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstMatroskamuxPad:video_0: caps = "video/x-vp8\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)10/1"
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/webm
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/webm\,\ streamheader\=\(buffer\)\<\ 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000502ad7b1830f42404d809f4753747265616d657220706c7567696e2076657273696f6e20312e342e31005741994753747265616d6572204d6174726f736b61206d7578657200446188062408b80e88c4001654ae6b010000000000003cae0100000000000033d7810183810173c588786b225315e5f279536e86566964656f00e00100000000000008b0820280ba8201e08686565f56503800\ \>"
看看 vp8parse source 似乎这个插件除了将 rtpvp8depay 提供的帧原封不动地传送到它的接收器之外什么也没做，但还将 src 上限设置为视频宽度、高度和帧率。
另一种可行的管道是：
gst-launch-1.0 -v -v -v -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 ! "application/x-rtp, payload=100, clock-rate=90000" ! rtpvp8depay ! vp8dec ! vp8enc ! webmmux streamable=true ! filesink location=/tmp/test.webm
但使用 vp8dec ！ vp8enc 显然没有多大意义，因为我已经收到了一个 vp8 编码流。
现在我的问题是如何在不重新编码流并且不依赖 vp8parse 的情况下解决这个问题？如果别无选择，我似乎必须使用它，但由于这是目前无法通过标准 gstreamer 插件包获得的插件，我想避免这种情况。是否可以将上限强制为特定的宽度、高度、帧率，这样 webmmux 就不会抱怨了？因为我认为这就是第一个管道未协商的原因。
我试过使用像 rtpvp8depay 这样的 capsfilter！ capsfilter 上限="video/x-vp8,width=640,height=480,framerate=10/1" ! webmmux 但它也不协商。
这是错误 747208，已在上游修复， 但是你系统中的 Gstreamer 版本可能是旧的 （Ubuntu 14.04 中的 1.2.4）并且仍然受到影响。
作为此类旧版本的解决方法，
如果您知道视频的帧大小
你可以在 depayloader 之后使用 capssetter 元素
手动设置 depayloader 遗漏的上限：
gst-launch-1.0 -v \
udpsrc multicast-group=224.1.1.1 auto-multicast=true port=1235 \
! "application/x-rtp, payload=100, clock-rate=90000" \
! rtpvp8depay ! capsetter caps="video/x-vp8,width=640,height=480" \
! webmmux streamable=true ! filesink location=/tmp/test.webm