生成的 MTLTexture 比 CGImage 更轻

Question

我有内核函数，它必须像 apple guide https://developer.apple.com/documentation/arkit/displaying_an_ar_experience_with_metal 中那样将 pixelBuffer(ARFrame.capturedImage) 创建的 Y 和 CbCr 纹理转换为 RGB 纹理但我克服了发光纹理

kernel void renderTexture(texture2d<float, access::sample> capturedImageTextureY [[ texture(0) ]],
                          texture2d<float, access::sample> capturedImageTextureCbCr [[ texture(1) ]],
                      
                          texture2d<float, access::read_write> outTextue [[texture(2)]],
                      
                      
                          uint2 size [[threads_per_grid]],
                          uint2 pid [[thread_position_in_grid]]){


constexpr sampler colorSampler(mip_filter::linear,
                               mag_filter::linear,
                               min_filter::linear);


const float4x4 ycbcrToRGBTransform = float4x4(
    float4(+1.0000f, +1.0000f, +1.0000f, +0.0000f),
    float4(+0.0000f, -0.3441f, +1.7720f, +0.0000f),
    float4(+1.4020f, -0.7141f, +0.0000f, +0.0000f),
    float4(-0.7010f, +0.5291f, -0.8860f, +1.0000f)
);

float2 texCoord;
texCoord.x = float(pid.x) / size.x;
texCoord.y = float(pid.y) / size.y;

// Sample Y and CbCr textures to get the YCbCr color at the given texture coordinate
float4 ycbcr = float4(capturedImageTextureY.sample(colorSampler,    texCoord).r,
                      capturedImageTextureCbCr.sample(colorSampler, texCoord).rg, 1.0);

float4 color = ycbcrToRGBTransform * ycbcr;

outTextue.write(color, pid);

}

我使用以下代码创建 CGImage：

var cgImage: CGImage?
VTCreateCGImageFromCVPixelBuffer(pixelBuffer, options: nil, imageOut: &cgImage)

cgImage有正常的闪电

当我尝试使用 MTKTextureLoader 从 cgImage 创建纹理时，我也忽略了光照纹理

如何像 cgImage 一样使用普通光获得 MTLTexture

cgImage:（预期结果）

内核函数：

使用此代码创建纹理：

let descriptor = MTLTextureDescriptor()
descriptor.width = Int(Self.maxTextureSize.width)
descriptor.height = Int(Self.maxTextureSize.height)
descriptor.usage = [.shaderWrite, .shaderRead]
    
let texture = MTLCreateSystemDefaultDevice()?.makeTexture(descriptor: descriptor)

并用内核函数写入像素。

已经尝试过 MTLTextureDescriptor 的不同像素格式

纹理加载器：

let textureLoader = MTKTextureLoader(device: MTLCreateSystemDefaultDevice()!)
let texturee = try! textureLoader.newTexture(cgImage: cgImage!, options: [.SRGB : (false as NSNumber)])

已经尝试过不同的 MTKTextureLoader.Options

GitHub 项目演示问题： PixelBufferToMTLTexture

Answer 1

when I try to create texture from cgImage with MTKTextureLoader I get over lighted texture

这是因为金属对你的纹理应用了伽马校正。

MTKTextureLoader有一个SRGB键，用于指定纹理数据是否存储为sRGB图像数据。

If the value is false, the image data is treated as linear pixel data. If the value is true, the image data is treated as sRGB pixel data. If this key is not specified and the image being loaded has been gamma-corrected, the image data uses the specified sRGB information.

let path = Bundle.main.path(forResource: "yourTexture", ofType: "png")!
let data = NSData(contentsOfFile: path) as! Data
let texture = try! textureLoader.newTexture(with: data, options: [MTKTextureLoaderOptionSRGB : (false as NSNumber)])

您也可以通过向着色器添加 gamma correcting equation 来解决此问题。

线性到 sRGB，反之亦然：

rgb = mix(rgb.0.0774, pow(rgb*0.9479 + 0.05213, 2.4), step(0.04045, rgb))

rgb = mix(rgb12.92, pow(rgb*0.4167) * 1.055 - 0.055, step(0.00313, rgb))

Answer 2

问题已解决，感谢 0xBFE1A8，通过添加伽玛校正

替换

outTextue.write(color, pid);

与：

outTextue.write(float4(pow(color.rgb, float3(2,2,2)), color.a), pid);

生成的 MTLTexture 比 CGImage 更轻

Resulting MTLTexture lighter than CGImage

textures

swift

metal

metalkit