来自 PhotoCaptureFrame 的 IMFMediaBuffer

Question

我有一个在 Unity 5.2 上使用 Hololens 可定位相机的计算机视觉插件。我使用 class UnityEngine.XR.WSA.WebCam.PhotoCaptureFrame 并以最高分辨率启用照片模式，即 2048x1156，NV12 颜色格式，这是相机原生格式。

PhotoCaptureFrame class : https://docs.unity3d.com/2017.2/Documentation/ScriptReference/XR.WSA.WebCam.PhotoCaptureFrame.html

目前，我这样做是为了让我的插件处理任何单张照片：

PhotoCaptureFrame photo; // created correctly through Unity API

List<byte> photoBuffer = new List<byte>();
photo.CopyRawImageDataIntoBuffer(photoBuffer);

// marshal photoBuffer to plugin (and it works fine for the computer vision part)

但是，它每次都会复制原始缓冲区。我想直接从我的插件中使用原始缓冲区。所以我尝试了这个：

IntPtr rawBufferPtr = photo.GetUnsafePointerToBuffer(); // COM pointer to IMFMediaBuffer is incremented
// send rawBufferPtr to plugin (native function described below)
Marshal.Release(rawBufferPtr);

IMFMediaBuffer 接口：https://msdn.microsoft.com/en-us/library/windows/desktop/ms696261(v=vs.85).aspx

在我的 C++ 插件中：

#include <Mfidl.h> // for IMFMediaBuffer support

void process_photo(void *photo_buffer_wrapper, int photo_width, int photo_height)
{
    // photo_buffer_wrapper = rawBufferPtr in managed environment

    IMFMediaBuffer *media_buffer = reinterpret_cast<IMFMediaBuffer *>(photo_buffer_wrapper);
    BYTE *photo_buffer = NULL;
    HRESULT result = media_buffer->Lock(&photo_buffer, NULL, NULL);
    if (SUCCEEDED(result))
    {
        // do image processing stuff here (with OpenCV) using photo_buffer
        media_buffer->Unlock;
    }
}

我觉得还不错。它也编译得很好。但是在运行的时候，我遇到了访问冲突并且应用程序在 Hololens 上崩溃了。

异常代码：0xC0000005

异常信息：线程试图读取或写入它没有适当访问权限的虚拟地址。

有人看到问题了吗？与我将 IMFMediaBuffer 对象从托管环境传递到非托管环境的方式有关吗？

非常感谢！

Answer 1

我会回答我自己的问题。

photo_buffer_wrapper并不是我想的指向IMFMediaBuffer的指针，而是指向IUnknown的指针。这是按预期工作的修改后的本机函数：

// UNMANAGED ENVIRONMENT
#include <Mfidl.h> // for IMFMediaBuffer support + other COM stuff

void process_photo(void *photo_buffer_unknown, int photo_width, int photo_height)
{
    // photo_buffer_unknown = photoCaptureFrame.GetUnsafePointerToBuffer() in managed environment which is an IntPtr

    IMFMediaBuffer *media_buffer;
    if (SUCCEEDED(reinterpret_cast<IUnknown *>(photo_buffer_unknown)->QueryInterface<IMFMediaBuffer>(&media_buffer)))
    {
        BYTE* photo_buffer = NULL;
        if (SUCCEEDED(media_buffer->Lock(&photo_buffer, NULL, NULL)))
        {
            // process photo_buffer with OpenCV (wrapped in a cv::Mat)

            media_buffer->Unlock();
            media_buffer->Release(); // QueryInterface on IUnknown has incremented reference count by one
        }
    }
}

注意：从 photoCaptureFrame.GetUnsafePointerToBuffer() 返回的指针仍然需要像我的问题一样在托管环境中释放：

// MANAGED ENVIRONMENT
IntPtr mediaBufferUnknownPtr = photoCaptureFrame.GetUnsafePointerToBuffer();
// send mediaBufferUnknownPtr to native function through extern DLL call
Marshal.Release(mediaBufferUnknownPtr)

来自 PhotoCaptureFrame 的 IMFMediaBuffer

IMFMediaBuffer from PhotoCaptureFrame

c#

c++

unity3d

com

hololens