使用 RealityKit 在相机和墙壁之间的对象上显示遮罩

Question

我制作了一个生成平面图的视频，其中如果用户离墙壁太近或者有任何物体挡在相机和 [=19 之间，我需要在特定位置同时捕捉墙壁和地板=] 然后需要在该对象上显示 Too Close 掩码，例如在 this video.

中显示

我尝试在 session(_ session: ARSession, didUpdate frame: ARFrame) 方法中使用 rycast 但我是 AR 的新手，不知道我们需要使用哪种方法。

func session(_ session: ARSession, didUpdate frame: ARFrame) {
        
            
        guard let query = self.arView?.makeRaycastQuery(from: self.arView?.center ?? CGPoint.zero,
                                                  allowing: .estimatedPlane,
                                                  alignment: .any)
        else { return }
        
        guard let raycastResult = self.arView?.session.raycast(query).first
        else { return }
        
        let currentPositionOfCamera = raycastResult.worldTransform.getPosition()
        if currentPositionOfCamera != .zero {
            let distanceFromCamera = frame.camera.transform.getPosition().distanceFrom(position: currentPositionOfCamera)
            print("Distance from raycast:",distanceFromCamera)
            if (distance < 0.5) {
                 print("Too Close")
            }
        }

    }

Answer 1

我也在学习 ARKit 和 RealityKit，但你的代码不会是：

let currentPositionOfCamera = self.arView.cameraTransform.translation

if currentPositionOfCamera != .zero {

    // distance is defined in simd as the distance between 2 points
    let distanceFromCamera = distance(raycastResult.worldTransform.position, currentPositionOfCamera)
    print("Distance from raycast:",distanceFromCamera)
    if (distanceFromCamera < 0.5) {
        print("Too Close")

        let rayDirection = normalize(raycastResult.worldTransform.position - self.arView.cameraTransform.translation)
        // This pulls the text back toward the camera from the plane
        let textPositionInWorldCoordinates = result.worldTransform.position - (rayDirection * 0.1)

        let textEntity = self.model(for: classification)
        // This scales the text so it is of a consistent size
        textEntity.scale = .one * raycastDistance

        var textPositionWithCameraOrientation = self.arView.cameraTransform
        textPositionWithCameraOrientation.translation = textPositionInWorldCoordinates
        // self.textAnchor is defined somewhere in the class as an optional
        self.textAnchor = AnchorEntity(world: textPositionWithCameraOrientation.matrix)
        textAnchor.addChild(textEntity)
        self.arView.scene.addAnchor(textAnchor)
    } else {
        guard let textAnchor = self.textAnchor else { return }
        self.removeAnchor(textAnchor)
    }
}

// Creates a text ModelEntity 
func tooCloseModel() -> ModelEntity {
        let lineHeight: CGFloat = 0.05
        let font = MeshResource.Font.systemFont(ofSize: lineHeight)
        let textMesh = MeshResource.generateText("Too Close", extrusionDepth: Float(lineHeight * 0.1), font: font)
        let textMaterial = SimpleMaterial(color: classification.color, isMetallic: true)
        let model = ModelEntity(mesh: textMesh, materials: [textMaterial])
        // Center the text
        model.position.x -= model.visualBounds(relativeTo: nil).extents.x / 2
        return model
}

此代码改编自Apple's Visualizing Scene Semantics。

使用 RealityKit 在相机和墙壁之间的对象上显示遮罩

show masking on object which is between camera and wall using RealityKit

raycasting

swift

arkit