检测 MediaStreamTrack 是否为 black/blank

Question

我正在使用 peerjs 创建视频聊天。

我正在使用以下功能切换相机 (on/off)：

function toggleCamera() {
    localStream.getVideoTracks()[0].enabled = !(localStream.getVideoTracks()[0].enabled);
}

调用此函数后，视频变黑，接收器只是黑屏（按预期工作）。现在我想检测 black/blank 屏幕，这样我就可以向用户显示一些消息或图标，表明相机已禁用并且没有流。

我该如何检测？

Answer 1

一段时间后我找到了解决方案：

var previousBytes = 0;
var previousTS = 0;
var currentBytes = 0;
var currentTS = 0;

// peer - new Peer()
// stream - local camera stream (received from navigator.mediaDevices.getUserMedia(constraints))
let connection = peer.call(peerID, stream);

// peerConnection - reference to RTCPeerConnection (https://peerjs.com/docs.html#dataconnection-peerconnection)
connection.peerConnection.getStats(null).then(stats => {
    stats.forEach(report => {
        if (report.type === "inbound-rtp") {

            currentBytes = report.bytesReceived;
            currentTS = report.timestamp;

            if (previousBytes == 0) {
                previousBytes = currentBytes;
                previousTS = currentTS;
                return;
            }

            console.log({ previousBytes })
            console.log({ currentBytes })

            var deltaBytes = currentBytes - previousBytes;
            var deltaTS = currentTS - previousTS;

            console.log("Delta: " + (deltaBytes / deltaTS) + " kB/s")
            previousBytes = currentBytes;
            previousTS = currentTS;

        }
    });
});

这段代码实际上是在每秒被调用的函数中。当相机打开且未被覆盖时，deltaBytes 在 100 到 250 之间，当相机关闭（以编程方式）或被覆盖（用餐巾纸或其他东西）时，因此相机流为 black/blank，deltaBytes 为 1.5- 3kbps。重新打开相机后，deltaBytes 出现峰值，达到 500kbps 左右。

这是简短的控制台日志：

    124.52747252747253 kB/s
    202.213 kB/s
    194.64764764764766 kB/s
    15.313 kB/s (this is where camera is covered)
    11.823823823823824 kB/s
    11.862137862137862 kB/s
    2.164 kB/s
    2.005 kB/s
    2.078078078078078 kB/s
    1.99 kB/s
    2.059 kB/s
    1.992992992992993 kB/s
    159.89810189810188 kB/s (uncovered camera)
    502.669 kB/s
    314.7927927927928 kB/s
    255.0909090909091 kB/s
    220.042 kB/s
    213.46353646353646 kB/s

编辑：所以最后我照@Philipp Hancke 说的做了。我创建了主连接，该连接从页面加载时一直打开到用户关闭它。通过此连接，我正在发送发起视频通话、取消视频会话、转动 on/off 摄像头等命令，...然后在另一端解析这些命令并执行功能。

function sendMutedMicCommand() { masterConnection.send(`${commands.MutedMic}`); }
function sendUnmutedMicCommand() { masterConnection.send(`${commands.UnmutedMic}`); }
function sendPromptVideoCallCommand() { masterConnection.send(`${commands.PromptVideoCall}`); }
function sendAcceptVideoCallCommand() { masterConnection.send(`${commands.AcceptVideoCall}`); }
function sendDeclineVideoCallCommand() { masterConnection.send(`${commands.DeclineVideoCall}`); }

Function which handles data:
function handleData(data) {
    let actionType = data;
    switch (actionType) {
        case commands.MutedMic: ShowMuteIconOnReceivingVideo(true); break;
        case commands.UnmutedMic: ShowMuteIconOnReceivingVideo(false); break;
        case commands.PromptVideoCall: showVideoCallModal(); break;
        case commands.AcceptVideoCall: startVideoConference(); break;
        case commands.DeclineVideoCall: showDeclinedCallAlert(); break;
        default: break;
    }
}


const commands = {
    MutedMic: "mutedMic",
    UnmutedMic: "unmutedMic",
    PromptVideoCall: "promptVideoCall",
    AcceptVideoCall: "acceptVideoCall",
    DeclineVideoCall: "declineVideoCall",
}

然后当我收到 mutedMic 命令时，我会显示带有交叉麦克风的图标。当我收到 AcceptVideoCall 命令时，我创建了另一个具有随机 ID 的对等点 videoCallPeer，然后将其发送到另一端。另一方然后创建另一个具有随机 ID 的对等点并使用收到的 ID 发起视频会话。

Answer 2

常见的方法是发送信令消息（通过正常路径或数据通道）。轮询 getStats 来检测黑帧是一种有效的方法，但在计算方面更昂贵。

检测 MediaStreamTrack 是否为 black/blank

Detect if MediaStreamTrack is black/blank

peer

webrtc

peerjs