在给定 2D 图像点、深度图和相机校准矩阵的情况下提取 3D 坐标
Extracting 3D coordinates given 2D image points, depth map and camera calibration matrices
我有一组 2D image 从 OpenCV FAST 角检测函数输出的关键点。使用 Asus Xtion I 也有一个时间同步的深度图，其中所有相机校准参数都是已知的。使用此信息，我想在 OpenCV.
中提取一组 3D 坐标（点云）
任何人都可以给我任何关于如何做的指示吗？提前致谢！
Nicolas Burrus 为像 Kinect 这样的深度传感器创建了一个很棒的教程。
http://nicolas.burrus.name/index.php/Research/KinectCalibration
我将复制并粘贴最重要的部分：
Mapping depth pixels with color pixels
The first step is to undistort rgb and depth images using the
estimated distortion coefficients. Then, using the depth camera
intrinsics, each pixel (x_d,y_d) of the depth camera can be projected
to metric 3D space using the following formula:
P3D.x = (x_d - cx_d) * depth(x_d,y_d) / fx_d
P3D.y = (y_d - cy_d) * depth(x_d,y_d) / fy_d
P3D.z = depth(x_d,y_d)
with fx_d, fy_d, cx_d and cy_d the intrinsics of the depth camera.
如果您对立体映射（kinect 的值）更感兴趣：
We can then reproject each 3D point on the color image and get its
color:
P3D' = R.P3D + T
P2D_rgb.x = (P3D'.x * fx_rgb / P3D'.z) + cx_rgb
P2D_rgb.y = (P3D'.y * fy_rgb / P3D'.z) + cy_rgb
with R and T the rotation and translation parameters estimated during
the stereo calibration.
The parameters I could estimate for my Kinect are:
Color
fx_rgb 5.2921508098293293e+02
fy_rgb 5.2556393630057437e+02
cx_rgb 3.2894272028759258e+02
cy_rgb 2.6748068171871557e+02
k1_rgb 2.6451622333009589e-01
k2_rgb -8.3990749424620825e-01
p1_rgb -1.9922302173693159e-03
p2_rgb 1.4371995932897616e-03
k3_rgb 9.1192465078713847e-01
Depth
fx_d 5.9421434211923247e+02
fy_d 5.9104053696870778e+02
cx_d 3.3930780975300314e+02
cy_d 2.4273913761751615e+02
k1_d -2.6386489753128833e-01
k2_d 9.9966832163729757e-01
p1_d -7.6275862143610667e-04
p2_d 5.0350940090814270e-03
k3_d -1.3053628089976321e+00
Relative transform between the sensors (in meters)
R [ 9.9984628826577793e-01, 1.2635359098409581e-03, -1.7487233004436643e-02,
-1.4779096108364480e-03, 9.9992385683542895e-01, -1.2251380107679535e-02,
1.7470421412464927e-02, 1.2275341476520762e-02, 9.9977202419716948e-01 ]
T [ 1.9985242312092553e-02, -7.4423738761617583e-04, -1.0916736334336222e-02 ]
我有一组 2D image 从 OpenCV FAST 角检测函数输出的关键点。使用 Asus Xtion I 也有一个时间同步的深度图，其中所有相机校准参数都是已知的。使用此信息，我想在 OpenCV.
3D 坐标（点云）
任何人都可以给我任何关于如何做的指示吗？提前致谢！
Nicolas Burrus 为像 Kinect 这样的深度传感器创建了一个很棒的教程。
http://nicolas.burrus.name/index.php/Research/KinectCalibration
我将复制并粘贴最重要的部分：
Mapping depth pixels with color pixels
The first step is to undistort rgb and depth images using the estimated distortion coefficients. Then, using the depth camera intrinsics, each pixel (x_d,y_d) of the depth camera can be projected to metric 3D space using the following formula:
P3D.x = (x_d - cx_d) * depth(x_d,y_d) / fx_d P3D.y = (y_d - cy_d) * depth(x_d,y_d) / fy_d P3D.z = depth(x_d,y_d)with fx_d, fy_d, cx_d and cy_d the intrinsics of the depth camera.
如果您对立体映射（kinect 的值）更感兴趣：
We can then reproject each 3D point on the color image and get its color:
P3D' = R.P3D + T P2D_rgb.x = (P3D'.x * fx_rgb / P3D'.z) + cx_rgb P2D_rgb.y = (P3D'.y * fy_rgb / P3D'.z) + cy_rgbwith R and T the rotation and translation parameters estimated during the stereo calibration.
The parameters I could estimate for my Kinect are:
Color
fx_rgb 5.2921508098293293e+02
fy_rgb 5.2556393630057437e+02
cx_rgb 3.2894272028759258e+02
cy_rgb 2.6748068171871557e+02
k1_rgb 2.6451622333009589e-01
k2_rgb -8.3990749424620825e-01
p1_rgb -1.9922302173693159e-03
p2_rgb 1.4371995932897616e-03
k3_rgb 9.1192465078713847e-01
Depth
fx_d 5.9421434211923247e+02
fy_d 5.9104053696870778e+02
cx_d 3.3930780975300314e+02
cy_d 2.4273913761751615e+02
k1_d -2.6386489753128833e-01
k2_d 9.9966832163729757e-01
p1_d -7.6275862143610667e-04
p2_d 5.0350940090814270e-03
k3_d -1.3053628089976321e+00
Relative transform between the sensors (in meters)
R [ 9.9984628826577793e-01, 1.2635359098409581e-03, -1.7487233004436643e-02,
-1.4779096108364480e-03, 9.9992385683542895e-01, -1.2251380107679535e-02,
1.7470421412464927e-02, 1.2275341476520762e-02, 9.9977202419716948e-01 ] T [ 1.9985242312092553e-02, -7.4423738761617583e-04, -1.0916736334336222e-02 ]