在 Metal 中绘制带纹理的矩形 - Swift

Question

我正在尝试使用 Swift 在 Metal 中绘制带纹理的 2D 矩形。所有在线示例（包括以下示例）都在 Objective-C 或 Swift 的旧版本中，并且很难移植到 Swift 5:

How to draw a textured rectangle with Metal https://developer.apple.com/documentation/metal/creating_and_sampling_textures

对我来说objective就是在视频的一角画一个简单的水印。我有一个 Renderer class 用于捕获帧，将它们绘制到我的金属视图（这很好用）然后想在顶部绘制一个徽标。我不能对相机帧使用相同的方法，因为它们是 YUV 格式，所以使用特殊的着色器。我的水印只是一个普通的旧 PNG 文件。

我可以绘制无纹理矩形，但尝试应用纹理失败 - 没有错误，没有任何问题。屏幕上什么也没有出现。当 Metal returns 没有错误来调试这些类型的问题时，我应该检查哪些关键事项？

这是一些代码：

func drawRect(renderEncoder: MTLRenderCommandEncoder, x:Double, y:Double, w:Double, h:Double, pipelineState:MTLRenderPipelineState, viewSize: CGSize)
    {
        //new viewport to size of image we want to display
        var viewPort = MTLViewport(originX: x, originY: y, width: w, height: h, znear: 0.0, zfar: 1.0)
        var viewPortSize = simd_float2(x: Float(w), y: Float(h))
        
        var logoQuadVertices:[AAPLVertex] = [
            //Triangle1: top left, top right, bottom left
            AAPLVertex(position: simd_float2(x: 0, y: 0), textureCoordinate: simd_float2(x: 0.0, y: 0.0)),
            AAPLVertex(position: simd_float2(x: 100, y: 0), textureCoordinate: simd_float2(x: 1.0, y: 0.0)),
            AAPLVertex(position: simd_float2(x: 0, y: 100), textureCoordinate: simd_float2(x: 0.0, y: 1.0)),
            //Triangle2: bottom left, bottom right, top right
            AAPLVertex(position: simd_float2(x: 0, y: 100), textureCoordinate: simd_float2(x: 0.0, y: 1.0)),
            AAPLVertex(position: simd_float2(x: 100, y: 100), textureCoordinate: simd_float2(x: 1.0, y: 1.0)),
            AAPLVertex(position: simd_float2(x: 100, y: 0), textureCoordinate: simd_float2(x: 1.0, y: 0.0))
        ]

        //Set the new viewport
        renderEncoder.setViewport(MTLViewport(originX: x, originY: y, width: w, height: h, znear: 0.0, zfar: 1.0))

        //Debug string
        renderEncoder.pushDebugGroup("DrawRect")

        //Set pipeline state
        renderEncoder.setRenderPipelineState(pipelineState)
    
        //Send vertex data of rectangle to shaders
        renderEncoder.setVertexBytes(logoQuadVertices, length: logoQuadVertices.count*MemoryLayout<AAPLVertex>.stride, index: Int(AAPLVertexInputIndexVertices.rawValue))

        //Send vertex bytes of viewport to shaders
        renderEncoder.setVertexBytes(&viewPortSize, length: MemoryLayout<simd_float2>.stride, index: Int(AAPLVertexInputIndexViewportSize.rawValue))

        // Set the texture
        renderEncoder.setFragmentTexture(texture, index: Int(kTextureIndexPNG.rawValue))

        //Sampler state
        if let samplerState = logoSamplerState {
            renderEncoder.setFragmentSamplerState(samplerState, index: 0)
        }
    
        //Draw the textured rectangle
        renderEncoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 6)
        
        //Debug
        renderEncoder.popDebugGroup()
    }

...
// Pipeline configuration
let logoPipelineStateDescriptor = MTLRenderPipelineDescriptor()
logoPipelineStateDescriptor.label = "MyLogoPipeline"
logoPipelineStateDescriptor.sampleCount = renderDestination.sampleCount
logoPipelineStateDescriptor.vertexFunction = 
defaultLibrary.makeFunction(name: "vertexShader")!
logoPipelineStateDescriptor.fragmentFunction = 
defaultLibrary.makeFunction(name: "samplingShader")
logoPipelineStateDescriptor.vertexDescriptor = logoPlaneVertexDescriptor
logoPipelineStateDescriptor.colorAttachments[0].pixelFormat = 
renderDestination.colorPixelFormat
logoPipelineStateDescriptor.depthAttachmentPixelFormat = 
renderDestination.depthStencilPixelFormat
logoPipelineStateDescriptor.stencilAttachmentPixelFormat = 
renderDestination.depthStencilPixelFormat
do {
    try logoPipelineState = device.makeRenderPipelineState(descriptor: logoPipelineStateDescriptor)
} catch let error {
    print("Failed to created logo pipeline state, error \(error)")
}
...

着色器：

typedef struct
{
    float4 position [[position]];
    float2 textureCoordinate;
} RasterizerData;

// Vertex Function
vertex RasterizerData
vertexShader(uint vertexID [[ vertex_id ]],
         constant AAPLVertex *vertexArray [[ buffer(AAPLVertexInputIndexVertices) ]],
         constant vector_uint2 *viewportSizePointer  [[ buffer(AAPLVertexInputIndexViewportSize) ]])

{
    RasterizerData out;

    float2 pixelSpacePosition = vertexArray[vertexID].position.xy;

    float2 viewportSize = float2(*viewportSizePointer);

    out.position = vector_float4(0.0, 0.0, 0.0, 1.0);
    out.position.xy = pixelSpacePosition / (viewportSize / 2.0);

    out.textureCoordinate = vertexArray[vertexID].textureCoordinate;

    return out;
}

// Fragment function
fragment float4
samplingShader(RasterizerData in [[stage_in]],
           texture2d<half> colorTexture [[ texture(AAPLTextureIndexBaseColor) ]])
{
    constexpr sampler textureSampler (mag_filter::linear,
                                  min_filter::linear);

    // Sample the texture to obtain a color
    const half4 colorSample = colorTexture.sample(textureSampler, in.textureCoordinate);

    // return the color of the texture
    return float4(colorSample);
}

Answer 1

根据@trojanfoe 的建议，调试器有一些用于调试 GPU 功能的有用工具。要访问此功能，请在运行您的应用程序中，在 XCode 中转到“调试”>“捕获 GPU 帧”。然后您可以探索层次结构，甚至可以查看顶点是如何加载的等等。

在我的例子中，顶点是错误的。我将它们作为像素坐标提供，而不是 -1.0、1.0 纹理坐标系。当我在调试器中的 verticies 数据没有按预期显示时，这一点很明显。您的问题可能是其他问题，但这是一个很好的起点。

针对我的问题修改代码：

var logoQuadVertices:[AAPLVertex] = [
        //Triangle1: top left, top right, bottom left
        AAPLVertex(position: simd_float2(x: -1.0, y: -1.0), textureCoordinate: simd_float2(x: 0.0, y: 0.0)),
        AAPLVertex(position: simd_float2(x: 1.0, y: -1.0), textureCoordinate: simd_float2(x: 1.0, y: 0.0)),
        AAPLVertex(position: simd_float2(x: -1.0, y: 1.0), textureCoordinate: simd_float2(x: 0.0, y: 1.0)),
        //Triangle2: bottom left, bottom right, top right
        AAPLVertex(position: simd_float2(x: -1.0, y: 1.0), textureCoordinate: simd_float2(x: 0.0, y: 1.0)),
        AAPLVertex(position: simd_float2(x: 1.0, y: 1.0), textureCoordinate: simd_float2(x: 1.0, y: 1.0)),
        AAPLVertex(position: simd_float2(x: 1.0, y: -1.0), textureCoordinate: simd_float2(x: 1.0, y: 0.0))
]

在 Metal 中绘制带纹理的矩形 - Swift

Draw a textured rectangle in Metal - Swift

ios

swift

metal

arkit

swift5