将 VNRectangleObservation 点转换为其他坐标系

Question

我需要转换 VNRectangleObservation 收到的 CGPoints (bottomLeft, bottomRight, topLeft, topRight) 到另一个坐标系（例如屏幕上视图的坐标）。

我定义一个请求：

    // Rectangle Request
    let rectangleDetectionRequest = VNDetectRectanglesRequest(completionHandler: handleRectangles)
    rectangleDetectionRequest.minimumSize = 0.5
    rectangleDetectionRequest.maximumObservations = 1

我在委托调用中从相机中获取了 sampleBuffer，并执行检测请求：

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {return}
    var requestOptions:[VNImageOption:Any] = [:]
    if let cameraIntrinsicData = CMGetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil) {
        requestOptions = [.cameraIntrinsics:cameraIntrinsicData]
    }
    let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: CGImagePropertyOrientation(rawValue:6)!, options: requestOptions)
    do {
        try imageRequestHandler.perform(self.requests)
    } catch {
        print(error)
    }

}

稍后在 completionHandler 中我收到结果：

func handleRectangles (request:VNRequest, error:Error?) {

     guard let results = request.results as? [VNRectangleObservation] else { return }

     let flipTransform = CGAffineTransform(scaleX: 1, y: -1).translatedBy(x: 0, y: -self.previewView.frame.height)
     let scaleTransform = CGAffineTransform.identity.scaledBy(x: self.previewView.frame.width, y: self.previewView.frame.height)

     for rectangle in results {
        let rectangleBounds = rectangle.boundingBox.applying(scaleTransform).applying(flipTransform)
        // convertedTopLeft = conversion(rectangle.topLeft) 
        // convertedTopRight = conversion(rectangle.topRight) 
        // convertedBottomLeft = conversion(rectangle.bottomLeft) 
        // convertedBottomRight = conversion(rectangle.bottomRight) 
    }
}

这适用于 CGRect 的 boundingBox，但我需要将 CGPoints 转换为另一个视图的坐标系。问题是我不知道如何从sampleBuffer: CMSampleBuffer的坐标系到previewView坐标系的转换。

谢谢！

Answer 1

假设你用的是相机图层，图层是AVCaptureVideoPreviewLayer。 (https://developer.apple.com/documentation/avfoundation/avcapturevideopreviewlayer)。所以如果要转换单点，使用这个函数：layerPointConverted (https://developer.apple.com/documentation/avfoundation/avcapturevideopreviewlayer/1623502-layerpointconverted)。请注意，由于 VNRectangleObservation 坐标系，y 是倒转的。

let convertedTopLeft: CGPoint = cameraLayer.layerPointConverted(fromCaptureDevicePoint: CGPoint(x: rectangle.topLeft.x, y: 1 - rectangle.topLeft.y))
let convertedTopRight: CGPoint = cameraLayer.layerPointConverted(fromCaptureDevicePoint: CGPoint(x: rectangle.topRight.x, y: 1 - rectangle.topRight.y))
let convertedBottomLeft: CGPoint = cameraLayer.layerPointConverted(fromCaptureDevicePoint: CGPoint(x: rectangle.bottomLeft.x, y: 1 - rectangle.bottomLeft.y))
let convertedBottomRight: CGPoint = cameraLayer.layerPointConverted(fromCaptureDevicePoint: CGPoint(x: rectangle.bottomRight.x, y: 1 - rectangle.bottomRight.y))

希望对您有所帮助

Answer 2

这只是将变换应用于 CGPoint 本身的问题，其中大小是我需要转置四个点的目标视图的 CGSize。

    let transform = CGAffineTransform.identity
        .scaledBy(x: 1, y: -1)
        .translatedBy(x: 0, y: -size.height)
        .scaledBy(x: size.width, y: size.height)

    let convertedTopLeft = rectangle.topLeft.applying(transform)
    let convertedTopRight = rectangle.topRight.applying(transform)
    let convertedBottomLeft = rectangle.bottomLeft.applying(transform)
    let convertedBottomRight = rectangle.bottomRight.applying(transform)

Answer 3

@mihaicris 的回答有效，但仅限于纵向模式。在景观中，我们需要做一些不同的事情。

if UIApplication.shared.statusBarOrientation.isLandscape {
    transform = CGAffineTransform.identity
        .scaledBy(x: -1, y: 1)
        .translatedBy(x: -size.width, y: 0)
        .scaledBy(x: size.width, y: size.height)
} else {
    transform = CGAffineTransform.identity
        .scaledBy(x: 1, y: -1)
        .translatedBy(x: 0, y: -size.height)
        .scaledBy(x: size.width, y: size.height)
}

let convertedTopLeft = rectangle.topLeft.applying(transform)
let convertedTopRight = rectangle.topRight.applying(transform)
let convertedBottomLeft = rectangle.bottomLeft.applying(transform)
let convertedBottomRight = rectangle.bottomRight.applying(transform)

将 VNRectangleObservation 点转换为其他坐标系

Convert VNRectangleObservation points to other coordinate system

computer-vision

ios

swift

apple-vision