将 ARFaceAnchor 与 SpriteKit 叠加层对齐

Question

我正在尝试计算特定几何点上的 SpriteKit 覆盖内容位置（不仅仅是覆盖视觉内容）ARFaceGeometry/ARFaceAnchor。

我正在使用计算出的世界坐标中的 SCNSceneRenderer.projectPoint，但结果是 y 轴倒置并且未与相机图像对齐：

let vertex4 = vector_float4(0, 0, 0, 1)
let modelMatrix = faceAnchor.transform
let world_vertex4 = simd_mul(modelMatrix, vertex4)
let pt3 = SCNVector3(x: Float(world_vertex4.x),
                     y: Float(world_vertex4.y),
                     z: Float(world_vertex4.z))
let sprite_pt = renderer.projectPoint(pt3)

// To visualize sprite_pt
let dot = SKSpriteNode(imageNamed: "dot")
dot.size = CGSize(width: 7, height: 7)
dot.position = CGPoint(x: CGFloat(sprite_pt.x), 
                       y: CGFloat(sprite_pt.y))
overlayScene.addChild(dot)

Answer 1

根据我的经验，ARKit 的 projectPoint 函数给出的屏幕坐标在绘图时可以直接使用，例如 CALayer。这意味着它们遵循 iOS 坐标，如 here 所述，其中原点在左上角，y 倒转。

SpriteKit 有 its own coordinate system:

The unit coordinate system places the origin at the bottom left corner of the frame and (1,1) at the top right corner of the frame. A sprite’s anchor point defaults to (0.5,0.5), which corresponds to the center of the frame.

最后，SKNodes被放置在一个原点在左下角的SKScene中。您应该确保您的 SKScene 与您的实际视图大小相同，否则原点可能不在视图的左下角，因此您从视图坐标定位节点可能不正确。的答案可能会有所帮助，特别是检查视图的 AspectFit 或 AspectFill 以确保缩小场景。

The Scene's origin is in the bottom left and depending on your scene size and scaling it may be off screen. This is where 0,0 is. So every child you add will start there and work its way right and up based on position. A SKSpriteNode has its origin in the center.

因此，从视图坐标和 SpriteKit 坐标转换的两个基本步骤是 1) 反转 y 轴，使原点位于左下角，以及 2) 确保您的 SKScene 框架与您的视图框架匹配。

我可以稍后更全面地测试一下，如果有任何问题，我会进行编辑

Answer 2

找到使用 camera.projectPoint 而不是 renderer.projectPoint 的有效转换。

要在 spritekit 上正确缩放点：设置 scaleMode=.aspectFill

我更新了 https://github.com/AnsonT/ARFaceSpriteKitMapping 来演示这个。

    guard let faceAnchor = anchor as? ARFaceAnchor,
        let camera = sceneView.session.currentFrame?.camera,
        let sie = overlayScene?.size
    else { return }

    let modelMatrix = faceAnchor.transform

    let vertices = faceAnchor.geometry.vertices

    for vertex in vertices {
        let vertex4 = vector_float4(vertex.x, vertex.y, vertex.z, 1)
        let world_vertex4 = simd_mul(modelMatrix, vertex4)
        let world_vector3 = simd_float3(x: world_vertex4.x, y: world_vertex4.y, z: world_vertex4.z)

        let pt = camera.projectPoint(world_vector3, orientation: .portrait, viewportSize: size)

        let dot = SKSpriteNode(imageNamed: "dot")
        dot.size = CGSize(width: 7, height: 7)
        dot.position = CGPoint(x: CGFloat(pt.x), y: size.height - CGFloat(pt.y))
        overlayScene?.addChild(dot)

    }

将 ARFaceAnchor 与 SpriteKit 叠加层对齐

Aligning ARFaceAnchor with SpriteKit overlay

scenekit

sprite-kit

swift

face

arkit