如何将 MTLTexture 转换为 CVpixelBuffer 以写入 AVAssetWriter？

Question

我需要在实时视频上应用滤镜，我正尝试在 Metal 中进行。

但是我在将过滤器编码到目标过滤器后将 MTLTexture 转换为 CVPixelBuffer 时遇到了问题。参考 (https://github.com/oklyc/MetalCameraSample-master-2)

这是我的代码。

if let pixelBuffer = pixelBuffer {
                CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags.init(rawValue: 0))             
                let region = MTLRegionMake2D(0, 0, Int(currentDrawable.layer.drawableSize.width), Int(currentDrawable.layer.drawableSize.height))                    
                let bytesPerPixel = 4;
                let bytesPerRow = CGFloat(bytesPerPixel) * currentDrawable.layer.drawableSize.width

                let tempBuffer = CVPixelBufferGetBaseAddress(pixelBuffer)
                destinationTexture.getBytes(tempBuffer!, bytesPerRow: Int(bytesPerRow), from: region1, mipmapLevel: 0)

                let image = self.imageFromCVPixelBuffer(buffer: pixelBuffer)
                CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags.init(rawValue: 0))

            }

imageFromCVPixelBuffer 方法如下所示。

func imageFromCVPixelBuffer(buffer: CVPixelBuffer) -> UIImage {

    let ciimage = CIImage(cvPixelBuffer: buffer)
    let context = CIContext(options: nil)
    let cgimgage = context.createCGImage(ciimage, from: CGRect(x: 0, y: 0, width: CVPixelBufferGetWidth(buffer), height: CVPixelBufferGetHeight(buffer)))

    let uiimage = UIImage(cgImage: cgimgage!)

    return uiimage
}

这是通过金属渲染图像的屏幕截图

这是将 MTLTexture 转换为 CVPixelBuffer 的同一图像的屏幕截图。

需要将 MTLtexture 转换为 CVPixelBuffer 才能写入 AVAssetWriter，然后将其保存到库中。

Answer 1

不要这样计算 bytesPerRow 自己。它被传递给 Metal，让 Metal 知道如何排列行。您希望 Metal 按照 CVPixelBuffer 期望的方式排列它们。因此，您应该使用CVPixelBufferGetBytesPerRow()来确定该值。

如何将 MTLTexture 转换为 CVpixelBuffer 以写入 AVAssetWriter？

How to convert a MTLTexture to CVpixelBuffer to write into an AVAssetWriter?

ios

swift

metal

metalkit

metal-performance-shaders