由于缓冲区不足，从 AVCaptureDataOutputSynchronizerDelegate 中适当释放缓冲区

Question

我正在使用 AVCaptureDataOutputSynchronizerDelegate 来处理视频、深度和元数据的捕获数据

    private let videoDataOutput = AVCaptureVideoDataOutput()
    private let depthDataOutput = AVCaptureDepthDataOutput()
    private let metadataOutput = AVCaptureMetadataOutput()

所以使用下面的代码，我可以在 AVCaptureDataOutputSynchronizerDelegate.

使用的委托方法中获取特定的视频数据

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {

    guard let syncedVideoData = synchronizedDataCollection.synchronizedData(for: self.videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { return }

问题是，当我尝试将 videoData 保存到如下数组中时，出现 OutOfBuffers 错误。如果我尝试保存与此数据相关的 videoData/the 图像 associated/anything，此问题仍然存在。

let array:[CMSampleBuffer] = []

...

array.append(syncedVideoData)
//Gets to about 5-6 sets of data, then it runs out of buffers. 
//I think the buffer is being retained permanently since I am saving to a global variable here.
//Leading to out of buffer error

所以，我想发生的事情是 因为我将任何相关数据保存到一个数组中，所以它会将数据保留在内存中的缓冲区中，而它通常会被释放。

webpage linked earlier for OutOfBuffers表示我可以

If you need to perform extended processing of captured data, copy that data into buffers whose lifetimes you manage instead of relying on buffers vended by the capture output.

我试图创建一个新的 CMSampleBuffer

extension VideoCapture: AVCaptureDataOutputSynchronizerDelegate {

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {

    var newData:CMSampleBuffer?

    guard let syncedVideoData = synchronizedDataCollection.synchronizedData(for: self.videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { return }
    guard !syncedVideoData.sampleBufferWasDropped else {
        print(syncedVideoData.droppedReason.rawValue)
        return
    }
    let videoSampleBuffer = syncedVideoData.sampleBuffer

    CMSampleBufferCreateCopy(allocator: kCFAllocatorDefault, sampleBuffer: videoSampleBuffer, sampleBufferOut: &newData)
    if(newData != nil) {
        self.buffer.append(newData!)
    }
}

但这会导致相同的问题——videoData 仍保留在缓冲区中。我得到了大约 5-6 组视频数据，然后我就没有更多的数据了。

关于如何 the outOfBuffers website"copy that data into buffers whose lifetimes you manage instead of relying on buffers vended by the capture output." 的任何指导？

Answer 1

我能够在 this buffer and this guide 和 Apple Docs 中的其他几个之后创建一个缓冲区。

...
guard let imagePixelBuffer = CMSampleBufferGetImageBuffer(videoSampleBuffer) else { fatalError() }


//First lock buffer
CVPixelBufferLockBaseAddress(imagePixelBuffer,
                                     CVPixelBufferLockFlags.readOnly)

//Do something with buffer
self.buffer = createMyBuffer(pixelBuffer: imagePixelBuffer)

//Unlock buffer
CVPixelBufferUnlockBaseAddress(imagePixelBuffer,
                                       CVPixelBufferLockFlags.readOnly)
self.doSomething(self.buffer)

...

func createMyBuffer(pixelBuffer: CVPixelBuffer) -> CVPixelBuffer? {
    let scaleWidth:Int = CVPixelBufferGetWidth(pixelBuffer)
    let scaleHeight:Int = CVPixelBufferGetHeight(pixelBuffer)

    let flags = CVPixelBufferLockFlags(rawValue: 0)
    guard kCVReturnSuccess == CVPixelBufferLockBaseAddress(pixelBuffer, flags) else {
        return nil
    }

    defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, flags) }

    guard let srcData = CVPixelBufferGetBaseAddress(pixelBuffer) else {
        print("Error: could not get pixel buffer base address")
        return nil
    }

    let srcBytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
    var srcBuffer = vImage_Buffer(data: srcData,
                                      height: vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer)),
                                      width: vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer)),
                                      rowBytes: srcBytesPerRow)

    let destBytesPerRow = scaleWidth*4
    guard let destData = malloc(scaleHeight*destBytesPerRow) else {
        print("Error: out of memory")
        return nil
    }

    var destBuffer = vImage_Buffer(data: destData,
                                       height: vImagePixelCount(scaleHeight),
                                       width: vImagePixelCount(scaleWidth),
                                       rowBytes: destBytesPerRow)

    let error = vImageScale_ARGB8888(&srcBuffer, &destBuffer, nil, vImage_Flags(kvImageLeaveAlphaUnchanged))
    if error != kvImageNoError {
        print("Error:", error)
        free(destData)
        return nil
    }

    let releaseCallback: CVPixelBufferReleaseBytesCallback = { _, ptr in
        if let ptr = ptr {
            free(UnsafeMutableRawPointer(mutating: ptr))
        }
    }

    let pixelFormat = CVPixelBufferGetPixelFormatType(pixelBuffer)
    var dstPixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferCreateWithBytes(nil, scaleWidth, scaleHeight,
                                                  pixelFormat, destData,
                                                  destBytesPerRow, releaseCallback,
                                                  nil, nil, &dstPixelBuffer)
    if status != kCVReturnSuccess {
        print("Error: could not create new pixel buffer")
        free(destData)
        return nil
    }
    return dstPixelBuffer
}

这有效 - 但似乎多余。我正在使用一个函数，我发现 "scales" 缓冲区，但我只是将它缩放到与当前缓冲区完全相同的大小，并且它 returns 一个我选择删除的新缓冲区。它是重复的，但功能有效。

由于缓冲区不足，从 AVCaptureDataOutputSynchronizerDelegate 中适当释放缓冲区

Appropriately Release Buffers From AVCaptureDataOutputSynchronizerDelegate Due to Out Of Buffers

swift

buffer

avcapture

cmsamplebuffer