WebRTC 到 Gstreamer 桥

Question

我正在尝试将音频从浏览器流式传输到服务器上的 gstreamer 管道。

我目前正在使用 Kurento，并修改 Hello World 示例以尝试将 RTP 端点连接到管道——但遇到了问题。

我知道媒体正在到达那里，因为当我交换记录端点时，我得到了有效的记录。

Kurento Node JS 是：

pipeline.create("RtpEndpoint", {}, function(error, rtpEndpoint) {
        if (error) {
            console.log("Recorder problem");
            return sendError(res, 500, error);
        }

        console.log("Creating WebRtcEndpoint");
        pipeline.create('WebRtcEndpoint', function(error, webRtcEndpoint) {
            if (error) {
                return sendError(res, 500, error);
            }

            console.log("Processing sdpOffer at server and generating sdpAnswer");
            webRtcEndpoint.processOffer(sdpOffer, function(error, sdpAnswer) {
                if (error) {
                    webRtcEndpoint.release();
                    return sendError(res, 500, error);
                }

                console.log("Connecting loopback");
                webRtcEndpoint.connect(webRtcEndpoint, function(error) {
                    if(error){
                        webRtcEndpoint.release();
                        return sendError(res, 500, error);
                    }
                    console.log("Sending sdpAnswer to client");
                    console.log(sdpAnswer);

                    webRtcEndpoint.connect(rtpEndpoint, function(error) {
                        if(error) {
                            webRtcEndpoint.release();
                            return sendError(res, 500, error);
                        }
                        rtpEndpoint.generateOffer(function(error, offer) {
                            fs.writeFile('/tmp/test.sdp',offer);
                            console.log("RTP OFFER GENERATED.");
                        });
                    });

                    res.type('application/sdp');
                    res.send(sdpAnswer);
                });     
            });
        });
    });

我的 GStreamer 管道是：

gst-launch-1.0 -vvvv filesrc location=/tmp/test.sdp ! sdpdemux ! decodebin ! autovideosink

哪个returns

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0-actual-sink-glimage': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSrc:udpsrc0: timeout = 10000000000
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSrc:udpsrc2: timeout = 10000000000
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:send_rtcp_src: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_0: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_0.GstProxyPad:proxypad4: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession1.GstPad:send_rtcp_src: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_1: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstUDPSink:udpsink1.GstPad:sink: caps = application/x-rtcp
/GstPipeline:pipeline0/GstSDPDemux:sdpdemux0/GstRtpBin:rtpbin0.GstGhostPad:send_rtcp_src_1.GstProxyPad:proxypad7: caps = application/x-rtcp
ERROR: from element /GstPipeline:pipeline0/GstSDPDemux:sdpdemux0: Could not read from resource.
Additional debug info:
gstsdpdemux.c(1213): gst_sdp_demux_handle_message (): /GstPipeline:pipeline0/GstSDPDemux:sdpdemux0:
Could not receive any UDP packets for 10.0000 seconds, maybe your firewall is blocking it.
Execution ended after 0:00:10.062018001
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

它不适用于 FFPMEG、VLC 等——结果类似于此处的 "Attempt 5.3"：https://altanaitelecom.wordpress.com/2015/02/26/continue-streaming-broadcasting-live-video-call-to-non-webrtc-supported-browsers-and-media-players/

我不认为存在防火墙问题，因为管道和 kurento 实例位于同一台虚拟机（没有防火墙）上——并且记录端点可以正常工作。链接不好吗？有没有更简单的方法？

Answer 1

使用 RtpEndpoint 比较棘手，因为您需要完成 de SDP 协商。这意味着

之后的某处

rtpEndpoint.generateOffer(...

你应该调用

rtpEndpoint.processAnswer(sdpAnswer, ...)

棘手的部分是您需要从您的 gstreamer 管道获取 sdpAnswer，如果您只想使用 gst-launch 获取它，这并不简单。最好的选择可能是编写一个小程序来创建管道并生成 sdpAnswer，以便您可以通过信号机制将其返回给 rtpEndpoint。

WebRTC 到 Gstreamer 桥

WebRTC to Gstreamer Bridge

gstreamer

sdp

webrtc

kurento