C#发送视频帧

Question

我想使用 UDP 连接从客户端向服务器发送视频帧。

如何从我的网络摄像头获取视频帧？我是否需要将每一帧都保存到我的计算机并从那里将其发送到服务器？如果我运行在 30fps 这意味着我需要在我的计算机上每秒保存 30 个视频帧？或者我可以直接从网络摄像头获取的视频帧发送吗？

目前我只能使用 Aforge 库打开网络摄像头并将视频显示到客户端 UI。

我浏览了许多不同的主题，但似乎没有从中受益，因为我的 C# 基础薄弱。

提前致谢！

Answer 1

您可以使用之前的 DirectShow.NET library (it's availabe as DirectShowLib over NuGet). There's a nice DxSnap sample that shows how to get still images. The hardest part is to get your head around the DirectShow API which may look foreign if you "have a weak foundation in C#" and/or haven't dealt with COM 代码逐帧捕获网络摄像头视频。

并且根据分辨率，由于 performance/bandwidth 限制，帧速率可能最高低于 30 fps。如果您需要在合适的分辨率下获得高帧率，您可能必须先使用一些比特率高效的编码器对视频进行编码。

这两个选项 - 逐帧和视频编码 - 都可以使用此库（尽管后者可能不容易通过 UDP 传递）。你只需要设置一个合适的图表。 GraphEditPlus 可能对此有很大帮助，特别是因为它可以为 DirectShow.NET 生成图表的 C# 代码（这里有点烦人的是，除非您购买应用程序）。

基本上，如果您确定发送每一帧，那么您只需构建一个如下图：视频捕获源 > 样本采集器 > 空渲染器。您将样本采集器设置为将每个帧重定向到您的 ISampleGrabberCB 实现，并且在 BufferCB 方法中，您将获得一个指向该帧位图数据的指针。这是一些我试图保持尽可能短的代码，从而牺牲了它的健壮性。看一看，但如果它有效，请不要将其留在生产中。

class FrameGrabber : ISampleGrabberCB
{
    IMediaControl mediaCtrl;
    int width, height, stride;

    public FrameGrabber(DsDevice camDevice)
    {
        IFilterGraph2 filterGraph;
        ICaptureGraphBuilder2 graphBuilder;
        IBaseFilter camBase, nullRenderer;
        ISampleGrabber sampleGrabber;

        filterGraph = new FilterGraph() as IFilterGraph2;
        mediaCtrl = filterGraph as IMediaControl;

        graphBuilder = new CaptureGraphBuilder2() as ICaptureGraphBuilder2;
        HRCheck(graphBuilder.SetFiltergraph(filterGraph));

        // Add camera
        HRCheck(filterGraph.AddSourceFilterForMoniker(
            camDevice.Mon, null, camDevice.Name, out camBase));

        // Add sample grabber
        sampleGrabber = new SampleGrabber() as ISampleGrabber;
        var mType = new AMMediaType()
        {
            majorType = MediaType.Video,
            subType = MediaSubType.RGB24,
            formatType = FormatType.VideoInfo
        };
        HRCheck(sampleGrabber.SetMediaType(mType));
        DsUtils.FreeAMMediaType(mType);
        HRCheck(sampleGrabber.SetCallback(this, 1));
        HRCheck(filterGraph.AddFilter(sampleGrabber as IBaseFilter, "CamGrabber"));

        // Add null renderer
        nullRenderer = new NullRenderer() as IBaseFilter;
        HRCheck(filterGraph.AddFilter(nullRenderer, "Null renderer"));

        // Render the webcam through the grabber and the renderer
        HRCheck(graphBuilder.RenderStream(PinCategory.Capture, MediaType.Video,
            camBase, sampleGrabber as IBaseFilter, nullRenderer));

        // Get resulting picture size
        mType = new AMMediaType();
        HRCheck(sampleGrabber.GetConnectedMediaType(mType));
        if (mType.formatType != FormatType.VideoInfo || mType.formatPtr == IntPtr.Zero)
        {
            throw new NotSupportedException("Unknown grabber media format");
        }
        var videoInfoHeader = Marshal.PtrToStructure(mType.formatPtr,
            typeof(VideoInfoHeader)) as VideoInfoHeader;
        width = videoInfoHeader.BmiHeader.Width;
        height = videoInfoHeader.BmiHeader.Height;
        Console.WriteLine("{0} x {1}", width, height); 
        stride = width * (videoInfoHeader.BmiHeader.BitCount / 8);
        DsUtils.FreeAMMediaType(mType);

        HRCheck(mediaCtrl.Run());
    }

    public int BufferCB(double SampleTime, IntPtr pBuffer, int BufferLen)
    {
        Console.WriteLine("BufferCB: " + SampleTime.ToString());
        // This is the bitmap of the frame but you may want
        // to copy it to some other memory location to
        // process/save/send it from there
        var bmp = new Bitmap(width, height, stride,
                PixelFormat.Format24bppRgb, pBuffer);
        return 0;
    }

    public int SampleCB(double SampleTime, IMediaSample pSample)
    {
        // This won't be called because sampleGrabber.SetCallback(this, 1)
        // -- 1 means BufferCB
        return Marshal.ReleaseComObject(pSample);
    }

    static void HRCheck(int hr)
    {
        DsError.ThrowExceptionForHR(hr);
    }

你会这样称呼它：

class Program
{
    static void Main(string[] args)
    {
        var cam = DsDevice.GetDevicesOfCat(FilterCategory.VideoInputDevice).First();
        var grabber = new FrameGrabber(cam);
        Console.ReadLine();
    }
}

C#发送视频帧

C# send video frame

c#

video

udp

frame