如何将 SCNRenderer 与现有的 MTLCommandBuffer 相结合？

Question

我通过将 SDK 提供的 OpenGL 上下文 (EAGLContext) 与 SceneKit 的实例 SCNRenderer 相结合，成功地将 Vuforia SDK Image Target Tracking feature 集成到 iOS 项目中。这使我能够利用 SceneKit 的 3D API 的简单性，同时受益于 Vuforia 的高精度图像检测。现在，我想通过用 Metal 替换 OpenGL 来做同样的事情。

一些背景故事

我能够使用 OpenGL 在 Vuforia 绘制的实时视频纹理之上绘制 SceneKit 对象，没有出现重大问题。

这是我在 OpenGL 中使用的简化设置：

func configureRenderer(for context: EAGLContext) {
    self.renderer = SCNRenderer(context: context, options: nil)
    self.scene = SCNScene()
    renderer.scene = scene

    // other scenekit setup
}

func render() {
    // manipulate scenekit nodes

    renderer.render(atTime: CFAbsoluteTimeGetCurrent())
}

Apple 在 iOS 12

上弃用了 OpenGL

自 Apple announced that it is deprecating OpenGL on iOS 12 以来，我认为尝试迁移此项目以使用 Metal 而不是 OpenGL.

是个好主意

理论上这应该很简单，因为 Vuforia 开箱即用地支持 Metal。但是，在尝试集成它时，我碰壁了。

问题

视图似乎只渲染 SceneKit 渲染器的结果，或 Vuforia 编码的纹理，但从未同时渲染两者。这取决于首先编码的内容。我需要做什么才能将两个结果混合在一起？

简而言之，这是有问题的设置：

func configureRenderer(for device: MTLDevice) {
    let renderer = SCNRenderer(device: device, options: nil)
    self.scene = SCNScene()
    renderer.scene = scene

    // other scenekit setup
}

func render(viewport: CGRect, commandBuffer: MTLCommandBuffer, drawable: CAMetalDrawable) {
    // manipulate scenekit nodes

    let renderPassDescriptor = MTLRenderPassDescriptor()
    renderPassDescriptor.colorAttachments[0].texture = drawable.texture
    renderPassDescriptor.colorAttachments[0].loadAction = .load
    renderPassDescriptor.colorAttachments[0].storeAction = .store
    renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(red: 0.0, green: 0, blue: 0, alpha: 0)

    renderer!.render(withViewport: viewport, commandBuffer: commandBuffer, passDescriptor: renderPassDescriptor)
}

我尝试在 encoder.endEncoding 之后或 commandBuffer.renderCommandEncoderWithDescriptor 之前调用 render :

metalDevice = MTLCreateSystemDefaultDevice();
metalCommandQueue = [metalDevice newCommandQueue];
id<MTLCommandBuffer>commandBuffer = [metalCommandQueue commandBuffer];

//// -----> call the `render(viewport:commandBuffer:drawable) here <------- \\

id<MTLRenderCommandEncoder> encoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];

// calls to encoder to render textures from Vuforia

[encoder endEncoding];

//// -----> or here <------- \\

[commandBuffer presentDrawable:drawable];
[commandBuffer commit];

在任何一种情况下，我都只能看到 SCNRenderer 或 encoder 的 结果，但不会在同一视图中看到两者。

在我看来，上面的编码传递和 SCNRenderer.render 正在覆盖彼此的缓冲区。

我在这里错过了什么？

Answer 1

我想我找到了答案。我在 scnrenderer 之后渲染 endEncoding，但我正在创建一个新的描述符。

    // Pass Metal context data to Vuforia Engine (we may have changed the encoder since
    // calling Vuforia::Renderer::begin)
    finishRender(UnsafeMutableRawPointer(Unmanaged.passRetained(drawable!.texture).toOpaque()), UnsafeMutableRawPointer(Unmanaged.passRetained(encoder!).toOpaque()))
    
    // ========== Finish Metal rendering ==========
    encoder?.endEncoding()
    
    // Commit the rendering commands
    // Command completed handler
    commandBuffer?.addCompletedHandler { _ in self.mCommandExecutingSemaphore.signal()}
    let screenSize = UIScreen.main.bounds.size
    let newDescriptor = MTLRenderPassDescriptor()
    
    // Draw to the drawable's texture
    newDescriptor.colorAttachments[0].texture = drawable?.texture

    // Store the data in the texture when rendering is complete
    newDescriptor.colorAttachments[0].storeAction = MTLStoreAction.store
    // Use textureDepth for depth operations.
    newDescriptor.depthAttachment.texture = mDepthTexture;
    renderer?.render(atTime: 0, viewport: CGRect(x: 0, y: 0, width: screenSize.width, height: screenSize.height), commandBuffer: commandBuffer!, passDescriptor: newDescriptor)
    
    // Present the drawable when the command buffer has been executed (Metal
    // calls to CoreAnimation to tell it to put the texture on the display when
    // the rendering is complete)
    commandBuffer?.present(drawable!)
    
    // Commit the command buffer for execution as soon as possible
    commandBuffer?.commit()

如何将 SCNRenderer 与现有的 MTLCommandBuffer 相结合？

How to combine SCNRenderer with an existing MTLCommandBuffer?

swift

opengl-es

scenekit

vuforia

metal