将 MetalView 与 SwiftUI 一起使用？我如何在其中放置要展示的东西？

Question

我被 SwiftUI 和 Metal 困住了，快要放弃了。

我从 https://developer.apple.com/forums/thread/119112?answerId=654964022#654964022 得到这个例子：

import MetalKit
struct MetalView: NSViewRepresentable {
    func makeCoordinator() -> Coordinator {
        Coordinator(self)
    }
    func makeNSView(context: NSViewRepresentableContext<MetalView>) -> MTKView {
        let mtkView = MTKView()
        mtkView.delegate = context.coordinator
        mtkView.preferredFramesPerSecond = 60
        mtkView.enableSetNeedsDisplay = true
        if let metalDevice = MTLCreateSystemDefaultDevice() {
            mtkView.device = metalDevice
        }
        mtkView.framebufferOnly = false
        mtkView.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0)
        mtkView.drawableSize = mtkView.frame.size
        mtkView.enableSetNeedsDisplay = true
        return mtkView
    }
    func updateNSView(_ nsView: MTKView, context: NSViewRepresentableContext<MetalView>) {
    }
    class Coordinator : NSObject, MTKViewDelegate {
        var parent: MetalView
        var metalDevice: MTLDevice!
        var metalCommandQueue: MTLCommandQueue!
        
        init(_ parent: MetalView) {
            self.parent = parent
            if let metalDevice = MTLCreateSystemDefaultDevice() {
                self.metalDevice = metalDevice
            }
            self.metalCommandQueue = metalDevice.makeCommandQueue()!
            super.init()
        }
        func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
        }
        func draw(in view: MTKView) {
            guard let drawable = view.currentDrawable else {
                return
            }
            let commandBuffer = metalCommandQueue.makeCommandBuffer()
            let rpd = view.currentRenderPassDescriptor
            rpd?.colorAttachments[0].clearColor = MTLClearColorMake(0, 1, 0, 1)
            rpd?.colorAttachments[0].loadAction = .clear
            rpd?.colorAttachments[0].storeAction = .store
            let re = commandBuffer?.makeRenderCommandEncoder(descriptor: rpd!)
            re?.endEncoding()
            commandBuffer?.present(drawable)
            commandBuffer?.commit()
        }
    }
}

...但我不知道如何使用这个 MetalView() 来显示数据，当我从 SwiftUI 视图调用它时它似乎确实有效。我想用它来显示一个 CIImage，它将被 CIFilters 过滤和操作...

有人可以为我指出正确的方向，告诉我如何告诉这个视图如何显示某些东西吗？我想我需要它来显示纹理的内容但是尝试了无数个小时并且最终从头开始无数次......

这就是我现在运行我的图像过滤器的方式，但它会导致滑块非常慢，这就是我决定尝试学习 Metal 的原因......但这真的很耗时而且。由于缺乏文档而令人沮丧...

func ciExposure (inputImage: CIImage, inputEV: Double) -> CIImage {
    let filter = CIFilter(name: "CIExposureAdjust")!
    filter.setValue(inputImage, forKey: kCIInputImageKey)
    filter.setValue(inputEV, forKey: kCIInputEVKey)
    return filter.outputImage!
}

我想我需要把那个 filter.outputImage 以某种方式传递给 MetalView？

非常非常感谢任何帮助...

Answer 1

我有一个小型 Core Image + SwiftUI 示例项目 on Github，这对您来说可能是一个很好的起点。它还没有涵盖很多内容，但它已经演示了如何显示过滤后的相机帧。

特别检查视图的draw function。它用于将 CIImage 渲染到 MTKView 中（您可以在委托的 draw 函数中执行相同的操作）。

Answer 2

好的，这对我有用：

func draw(in view: MTKView) {
            guard let drawable = view.currentDrawable else {
                return
            }
            
            let colorSpace = CGColorSpaceCreateDeviceRGB()

            let commandBuffer = metalCommandQueue.makeCommandBuffer()
            
            let rpd = view.currentRenderPassDescriptor
            rpd?.colorAttachments[0].clearColor = MTLClearColorMake(0, 1, 0, 1)
            rpd?.colorAttachments[0].loadAction = .clear
            rpd?.colorAttachments[0].storeAction = .store
            
            let re = commandBuffer?.makeRenderCommandEncoder(descriptor: rpd!)
            re?.endEncoding()
                
            context.render((AppState.shared.rawImage ?? AppState.shared.rawImageOriginal)!,
                to: drawable.texture,
                commandBuffer: commandBuffer,
                bounds: AppState.shared.rawImageOriginal!.extent,
                colorSpace: colorSpace)
            
            commandBuffer?.present(drawable)
            commandBuffer?.commit()
        }

AppState.shared.rawImage 是我从过滤函数得到的 CIImage 纹理。

上下文是在别处制作的，但应该是：

context = CIContext(mtlDevice: metalDevice)

接下来是添加 Frank Schlegel 提供的代码的居中部分。

将 MetalView 与 SwiftUI 一起使用？我如何在其中放置要展示的东西？

Use MetalView with SwiftUI? How do I put something to display in there?

macos

core-image

swift

metal

swiftui