从 MtlTexture 中获取字节以创建 CVPixelBufferRef

Question

尝试在 SCNRender 对象的每次调用渲染时从 MTLTexture 创建 CVPixelBufferRef：

  CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
        let bytesPerRow = 4 * Int(textureSizeX)
        let region = MTLRegionMake2D(0, 0, Int(textureSizeX), Int(textureSizeY))

    var tmpBuffer = CVPixelBufferGetBaseAddress(pixelBuffer!);

    offscreenTexture.getBytes(tmpBuffer!, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)

    CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))

然后转换成UIImage显示在屏幕上

    let ciImage = CIImage.init(cvPixelBuffer: pixelBuffer!)
    let temporaryContext = CIContext(options: nil)
    let tempImage = temporaryContext.createCGImage(ciImage, from: CGRect(x: 0, y: 0, width: textureSizeX, height: textureSizeY))
    let uiImage = UIImage.init(cgImage: tempImage!)

但是没有显示图片

Answer 1

你应该反过来想，创建一个使用 CoreVideo 缓冲区作为后备缓冲区的金属纹理对象，然后以正常方式渲染到纹理中。

- (id<MTLTexture>) makeBGRACoreVideoTexture:(CGSize)size
                        cvPixelBufferRefPtr:(CVPixelBufferRef*)cvPixelBufferRefPtr
{
  int width = (int) size.width;
  int height = (int) size.height;

  // CoreVideo pixel buffer backing the indexes texture

  NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                           [NSNumber numberWithBool:YES], kCVPixelBufferMetalCompatibilityKey,
                           [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                           [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                           nil];

  CVPixelBufferRef pxbuffer = NULL;

  CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
                                        width,
                                        height,
                                        kCVPixelFormatType_32BGRA,
                                        (__bridge CFDictionaryRef) options,
                                        &pxbuffer);

  NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);

  *cvPixelBufferRefPtr = pxbuffer;

  CVMetalTextureRef cvTexture = NULL;

  CVReturn ret = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                           _textureCache,
                                                           pxbuffer,
                                                           nil,
                                                           MTLPixelFormatBGRA8Unorm,
                                                           CVPixelBufferGetWidth(pxbuffer),
                                                           CVPixelBufferGetHeight(pxbuffer),
                                                           0,
                                                           &cvTexture);

  NSParameterAssert(ret == kCVReturnSuccess && cvTexture != NULL);

  id<MTLTexture> metalTexture = CVMetalTextureGetTexture(cvTexture);

  CFRelease(cvTexture);

  return metalTexture;
}

Answer 2

我遇到了同样的问题并找到了以下解决方案。它就像一个魅力:)

func image(from texture: MTLTexture) -> UIImage? {
    let bytesPerPixel = 4

    // The total number of bytes of the texture
    let imageByteCount = texture.width * texture.height * bytesPerPixel

    // The number of bytes for each image row
    let bytesPerRow = texture.width * bytesPerPixel

    // An empty buffer that will contain the image
    var src = [UInt8](repeating: 0, count: Int(imageByteCount))

    // Gets the bytes from the texture
    let region = MTLRegionMake2D(0, 0, texture.width, texture.height)
    texture.getBytes(&src, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)

    // Creates an image context
    let bitmapInfo = CGBitmapInfo(rawValue: (CGBitmapInfo.byteOrder32Big.rawValue | CGImageAlphaInfo.premultipliedLast.rawValue))
    let bitsPerComponent = 8
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let context = CGContext(data: &src, width: texture.width, height: texture.height, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue)

    // Creates the image from the graphics context
    guard let dstImage = context?.makeImage() else { return nil }

    // Creates the final UIImage
    return UIImage(cgImage: dstImage, scale: 0.0, orientation: .up)
}

来源：https://www.invasivecode.com/weblog/metal-image-processing 您可能还对以下内容感兴趣：https://github.com/hollance/CoreMLHelpers/tree/master/CoreMLHelpers

从 MtlTexture 中获取字节以创建 CVPixelBufferRef

getBytes from MtlTexture to create CVPixelBufferRef

ios

metal

cvpixelbuffer