从 ARCamera 设置 VNImageOptionCameraIntrinsics
Set VNImageOptionCameraIntrinsics from ARCamera
我正在构建一个结合了 ARKit 和 CoreML 的应用程序。我使用以下行将帧传递给 VNImageRequestHandler
:
// the frame of the current Scene
CVPixelBufferRef pixelBuffer = _cameraPreview.session.currentFrame.capturedImage;
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer options:requestOptions];
注意 requestOptions
。它应该包含 VNImageOptionCameraIntrinsics
字段,该字段将相机内在函数传递给 CoreML。
在使用 ARKit 之前,我使用 CMSampleBufferRef
从相机获取图像。可以使用以下方法检索和设置内部函数:
CFTypeRef cameraIntrinsicData = CMGetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil);
requestOptions[VNImageOptionCameraIntrinsics] = (__bridge id)(cameraIntrinsicData);
但是,我现在使用的是 ARFrame
,但由于 pixelBuffer
已旋转,我仍想设置正确的内在函数。
查看文档:
https://developer.apple.com/documentation/vision/vnimageoption?language=objc
https://developer.apple.com/documentation/arkit/arcamera/2875730-intrinsics?language=objc
我们可以看到 ARCamera
也提供了内在函数,但是,如何在 requestOptions
中正确设置此值?
到目前为止应该是这样的:
ARCamera *camera = _cameraPreview.session.currentFrame.camera;
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
// How to put camera.intrinsics here?
requestOptions[VNImageOptionCameraIntrinsics] = camera.intrinsics;
正如评论中提到的 Giovanni,将 UIDeviceOrientation
转换为 CGImagePropertyOrientation
避免了使用 VNImageOptionCameraIntrinsics
:
+(CGImagePropertyOrientation) getOrientation {
CGImagePropertyOrientation orientation;
UIDeviceOrientation deviceOrientation = [[UIDevice currentDevice] orientation];
switch (deviceOrientation) {
case UIDeviceOrientationPortrait:
orientation = kCGImagePropertyOrientationRight;
break;
case UIDeviceOrientationPortraitUpsideDown:
orientation = kCGImagePropertyOrientationLeft;
break;
case UIDeviceOrientationLandscapeLeft:
orientation = kCGImagePropertyOrientationUp;
break;
case UIDeviceOrientationLandscapeRight:
orientation = kCGImagePropertyOrientationDown;
break;
default:
orientation = kCGImagePropertyOrientationRight;
break;
}
return orientation;
}
- (void)captureOutput {
ARFrame *frame = self.cameraPreview.session.currentFrame;
CVPixelBufferRef pixelBuffer = frame.capturedImage;
CGImagePropertyOrientation deviceOrientation = [Utils getOrientation];
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer orientation:deviceOrientation options:requestOptions];
[handler performRequests:@[[self request]] error:nil];
}
我正在构建一个结合了 ARKit 和 CoreML 的应用程序。我使用以下行将帧传递给 VNImageRequestHandler
:
// the frame of the current Scene
CVPixelBufferRef pixelBuffer = _cameraPreview.session.currentFrame.capturedImage;
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer options:requestOptions];
注意 requestOptions
。它应该包含 VNImageOptionCameraIntrinsics
字段,该字段将相机内在函数传递给 CoreML。
在使用 ARKit 之前,我使用 CMSampleBufferRef
从相机获取图像。可以使用以下方法检索和设置内部函数:
CFTypeRef cameraIntrinsicData = CMGetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil);
requestOptions[VNImageOptionCameraIntrinsics] = (__bridge id)(cameraIntrinsicData);
但是,我现在使用的是 ARFrame
,但由于 pixelBuffer
已旋转,我仍想设置正确的内在函数。
查看文档:
https://developer.apple.com/documentation/vision/vnimageoption?language=objc
https://developer.apple.com/documentation/arkit/arcamera/2875730-intrinsics?language=objc
我们可以看到 ARCamera
也提供了内在函数,但是,如何在 requestOptions
中正确设置此值?
到目前为止应该是这样的:
ARCamera *camera = _cameraPreview.session.currentFrame.camera;
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
// How to put camera.intrinsics here?
requestOptions[VNImageOptionCameraIntrinsics] = camera.intrinsics;
正如评论中提到的 Giovanni,将 UIDeviceOrientation
转换为 CGImagePropertyOrientation
避免了使用 VNImageOptionCameraIntrinsics
:
+(CGImagePropertyOrientation) getOrientation {
CGImagePropertyOrientation orientation;
UIDeviceOrientation deviceOrientation = [[UIDevice currentDevice] orientation];
switch (deviceOrientation) {
case UIDeviceOrientationPortrait:
orientation = kCGImagePropertyOrientationRight;
break;
case UIDeviceOrientationPortraitUpsideDown:
orientation = kCGImagePropertyOrientationLeft;
break;
case UIDeviceOrientationLandscapeLeft:
orientation = kCGImagePropertyOrientationUp;
break;
case UIDeviceOrientationLandscapeRight:
orientation = kCGImagePropertyOrientationDown;
break;
default:
orientation = kCGImagePropertyOrientationRight;
break;
}
return orientation;
}
- (void)captureOutput {
ARFrame *frame = self.cameraPreview.session.currentFrame;
CVPixelBufferRef pixelBuffer = frame.capturedImage;
CGImagePropertyOrientation deviceOrientation = [Utils getOrientation];
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer orientation:deviceOrientation options:requestOptions];
[handler performRequests:@[[self request]] error:nil];
}