什么会导致重复调用 MetalKit MTKView 的 draw() 函数出现延迟

What can cause lag in recurrent calls to the draw() function of a MetalKit MTKView

我正在使用 swift 4.0 MetalKit API for macOS 10.13 设计一个 Cocoa 应用程序。我在这里报告的所有内容都是在我的 2015 MBPro 上完成的。

我已经成功地实现了一个 MTKView,它可以很好地渲染具有低顶点数的简单几何体(立方体、三角形等)。我实现了一个基于鼠标拖动的相机,它可以旋转、扫射和放大。这是我旋转立方体时 xcode FPS 调试屏幕的屏幕截图:

但是,当我尝试加载仅包含约 1500 个顶点的数据集(每个顶点存储为 7 x 32 位浮点数...即:总共 42 kB)时,我的 FPS 开始变得非常滞后。我将在下面展示代码实现。这是一个屏幕截图(请注意,在此图像上,视图仅包含少数顶点,这些顶点呈现为大点):

这是我的实现:

1) viewDidLoad() :

override func viewDidLoad() {

    super.viewDidLoad()

    // Initialization of the projection matrix and camera
    self.projectionMatrix = float4x4.makePerspectiveViewAngle(float4x4.degrees(toRad: 85.0),
                                      aspectRatio: Float(self.view.bounds.size.width / self.view.bounds.size.height),
                                      nearZ: 0.01, farZ: 100.0)
    self.vCam = ViewCamera()

    // Initialization of the MTLDevice
    metalView.device = MTLCreateSystemDefaultDevice()
    device = metalView.device
    metalView.colorPixelFormat = .bgra8Unorm

    // Initialization of the shader library
    let defaultLibrary = device.makeDefaultLibrary()!
    let fragmentProgram = defaultLibrary.makeFunction(name: "basic_fragment")
    let vertexProgram = defaultLibrary.makeFunction(name: "basic_vertex")

    // Initialization of the MTLRenderPipelineState
    let pipelineStateDescriptor = MTLRenderPipelineDescriptor()
    pipelineStateDescriptor.vertexFunction = vertexProgram
    pipelineStateDescriptor.fragmentFunction = fragmentProgram
    pipelineStateDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
    pipelineState = try! device.makeRenderPipelineState(descriptor: pipelineStateDescriptor)

    // Initialization of the MTLCommandQueue
    commandQueue = device.makeCommandQueue()

    // Initialization of Delegates and BufferProvider for View and Projection matrix MTLBuffer
    self.metalView.delegate = self
    self.metalView.eventDelegate = self
    self.bufferProvider = BufferProvider(device: device, inflightBuffersCount: 3, sizeOfUniformsBuffer: MemoryLayout<Float>.size * float4x4.numberOfElements() * 2)
}

2) 为立方体顶点加载 MTLBuffer :

private func makeCubeVertexBuffer() {

    let cube = Cube()
    let vertices = cube.verticesArray
    var vertexData = Array<Float>()
    for vertex in vertices{
        vertexData += vertex.floatBuffer()
    }
    VDataSize = vertexData.count * MemoryLayout.size(ofValue: vertexData[0])
    self.vertexBuffer = device.makeBuffer(bytes: vertexData, length: VDataSize!, options: [])!
    self.vertexCount = vertices.count
}

3) 为数据集顶点加载 MTLBuffer。请注意,我明确声明此缓冲区的存储模式为 Private 以确保 GPU 有效访问数据,因为一旦加载缓冲区 CPU 就不需要访问数据。另外,请注意,我只加载了实际数据集中 1/100 的顶点,因为当我尝试完全加载它时,我机器上的整个 OS 开始滞后(只有 4.2 MB 的数据)。

public func loadDataset(datasetVolume: DatasetVolume) {

    // Load dataset vertices
    self.datasetVolume = datasetVolume
    self.datasetVertexCount = self.datasetVolume!.vertexCount/100
    let rgbaVertices = self.datasetVolume!.rgbaPixelVolume[0...(self.datasetVertexCount!-1)]
    var vertexData = Array<Float>()
    for vertex in rgbaVertices{
            vertexData += vertex.floatBuffer()
    }
    let dataSize = vertexData.count * MemoryLayout.size(ofValue: vertexData[0])

    // Make two MTLBuffer's: One with Shared storage mode in which data is initially loaded, and a second one with Private storage mode
    self.datasetVertexBuffer = device.makeBuffer(bytes: vertexData, length: dataSize, options: MTLResourceOptions.storageModeShared)
    self.datasetVertexBufferGPU = device.makeBuffer(length: dataSize, options: MTLResourceOptions.storageModePrivate)

    // Create a MTLCommandBuffer and blit the vertex data from the Shared MTLBuffer to the Private MTLBuffer
    let commandBuffer = self.commandQueue.makeCommandBuffer()
    let blitEncoder = commandBuffer!.makeBlitCommandEncoder()
    blitEncoder!.copy(from: self.datasetVertexBuffer!, sourceOffset: 0, to: self.datasetVertexBufferGPU!, destinationOffset: 0, size: dataSize)
    blitEncoder!.endEncoding()
    commandBuffer!.commit()

    // Clean up
    self.datasetLoaded = true
    self.datasetVertexBuffer = nil
}

4) 最后,这是渲染循环。同样,这是使用 MetalKit。

func draw(in view: MTKView) {
    render(view.currentDrawable)
}

private func render(_ drawable: CAMetalDrawable?) {
    guard let drawable = drawable else { return }

    // Make sure an MTLBuffer for the View and Projection matrices is available
    _ = self.bufferProvider?.availableResourcesSemaphore.wait(timeout: DispatchTime.distantFuture)

    // Initialize common RenderPassDescriptor
    let renderPassDescriptor = MTLRenderPassDescriptor()
    renderPassDescriptor.colorAttachments[0].texture = drawable.texture
    renderPassDescriptor.colorAttachments[0].loadAction = .clear
    renderPassDescriptor.colorAttachments[0].clearColor = Colors.White
    renderPassDescriptor.colorAttachments[0].storeAction = .store

    // Initialize a CommandBuffer and add a CompletedHandler to release an MTLBuffer from the BufferProvider once the GPU is done processing this command
    let commandBuffer = self.commandQueue.makeCommandBuffer()
    commandBuffer?.addCompletedHandler { (_) in
        self.bufferProvider?.availableResourcesSemaphore.signal()
    }

    // Update the View matrix and obtain an MTLBuffer for it and the projection matrix
    let camViewMatrix = self.vCam.getLookAtMatrix()
    let uniformBuffer = bufferProvider?.nextUniformsBuffer(projectionMatrix: projectionMatrix, camViewMatrix: camViewMatrix)

    // Initialize a MTLParallelRenderCommandEncoder
    let parallelEncoder = commandBuffer?.makeParallelRenderCommandEncoder(descriptor: renderPassDescriptor)

    // Create a CommandEncoder for the cube vertices if its data is loaded
    if self.cubeLoaded == true {
        let cubeRenderEncoder = parallelEncoder?.makeRenderCommandEncoder()
        cubeRenderEncoder!.setCullMode(MTLCullMode.front)
        cubeRenderEncoder!.setRenderPipelineState(pipelineState)
        cubeRenderEncoder!.setTriangleFillMode(MTLTriangleFillMode.fill)
        cubeRenderEncoder!.setVertexBuffer(self.cubeVertexBuffer, offset: 0, index: 0)
        cubeRenderEncoder!.setVertexBuffer(uniformBuffer, offset: 0, index: 1)
        cubeRenderEncoder!.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexCount!, instanceCount: self.cubeVertexCount!/3)
        cubeRenderEncoder!.endEncoding()
    }

    // Create a CommandEncoder for the dataset vertices if its data is loaded
    if self.datasetLoaded == true {
        let rgbaVolumeRenderEncoder = parallelEncoder?.makeRenderCommandEncoder()
        rgbaVolumeRenderEncoder!.setRenderPipelineState(pipelineState)
        rgbaVolumeRenderEncoder!.setVertexBuffer( self.datasetVertexBufferGPU!, offset: 0, index: 0)
        rgbaVolumeRenderEncoder!.setVertexBuffer(uniformBuffer, offset: 0, index: 1)
        rgbaVolumeRenderEncoder!.drawPrimitives(type: .point, vertexStart: 0, vertexCount: datasetVertexCount!, instanceCount: datasetVertexCount!)
        rgbaVolumeRenderEncoder!.endEncoding()
    }

    // End CommandBuffer encoding and commit task
    parallelEncoder!.endEncoding()
    commandBuffer!.present(drawable)
    commandBuffer!.commit()
}

好的,这些是我在尝试找出导致延迟的原因时所经历的步骤,请记住,延迟效果与数据集的顶点缓冲区的大小成正比:

  1. 我一开始虽然是因为 GPU 无法足够快地访问内存,因为它处于共享存储模式 -> 我将数据集 MTLBuffer 更改为私有存储模式。这并没有解决问题。

  2. 然后我认为问题是由于 CPU 在我的 render() 函数中花费了太多时间。这可能是由于 BufferProvider 的问题,或者可能是因为 CPU 试图以某种方式 reprocess/reload 数据集顶点缓冲每帧 -> 为了检查这一点,我使用了 Time Profiler xcode 的乐器。不幸的是,问题似乎是应用程序很少调用此渲染方法(换句话说,MTKView 的 draw() 方法)。以下是一些截图:

这就是我所在的地方。问题似乎是 CPU 以某种方式被这 42 kB 的数据超载......递归地。我还在 xcode 的 Instruments 中使用分配器进行了测试。据我所知,没有内存泄漏的迹象(您可能已经注意到其中很多对我来说都是新的)。

抱歉让您费解 post,我希望它不会太难理解。预先感谢大家的帮助。

编辑:

这是我的着色器,如果您想查看它们:

struct VertexIn{
    packed_float3 position;
    packed_float4 color;
};

struct VertexOut{
    float4 position [[position]];  
    float4 color;
    float  size [[point_size]];
};

struct Uniforms{
    float4x4 cameraMatrix;
    float4x4 projectionMatrix;
};


vertex VertexOut basic_vertex(const device VertexIn* vertex_array [[ buffer(0) ]],
                              constant Uniforms&  uniforms    [[ buffer(1) ]],
                              unsigned int vid [[ vertex_id ]]) {

    float4x4 cam_Matrix = uniforms.cameraMatrix;
    float4x4 proj_Matrix = uniforms.projectionMatrix;

    VertexIn VertexIn = vertex_array[vid];

    VertexOut VertexOut;
    VertexOut.position = proj_Matrix * cam_Matrix * float4(VertexIn.position,1);
    VertexOut.color = VertexIn.color;
    VertexOut.size = 15;

    return VertexOut;
}

fragment half4 basic_fragment(VertexOut interpolated [[stage_in]]) {
    return half4(interpolated.color[0], interpolated.color[1], interpolated.color[2], interpolated.color[3]);
}

我认为主要问题是您告诉 Metal 在不应该的情况下进行实例化绘图。这一行:

rgbaVolumeRenderEncoder!.drawPrimitives(type: .point, vertexStart: 0, vertexCount: datasetVertexCount!, instanceCount: datasetVertexCount!)

告诉 Metal 绘制每个 datasetVertexCount! 个顶点的 datasetVertexCount! 个实例。 GPU 工作量随着顶点数的平方增长。此外,由于您不使用实例 ID 来调整顶点位置,因此所有这些实例都是相同的,因此是多余的。

我认为这同样适用于这一行:

cubeRenderEncoder!.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexCount!, instanceCount: self.cubeVertexCount!/3)

虽然不清楚self.cubeVertexCount!是什么,是否随着vertexCount增长。在任何情况下,由于您似乎使用的是相同的管道状态,因此使用的是相同的着色器,但没有使用实例 ID,因此它仍然是无用且浪费的。

其他:

当您实际上并没有使用它启用的并行性时,为什么要使用 MTLParallelRenderCommandEncoder?不要那样做。

你在任何地方使用 MemoryLayoutsize 方法,你几乎肯定应该使用 stride 来代替。如果您正在计算复合数据结构的步幅,请 而不是 采用该结构的一个元素的步幅并乘以元素的数量。跨出整个数据结构的步伐