将 2D 点转换为 3D 位置
Converting 2D point to 3D location
我有一个已知 cameraMatrix
和 distCoeffs
的固定相机。我还有一个固定的棋盘,transform
和 rotation
向量也是使用 solvePnP
计算的。
我想知道如何在棋盘所在的同一平面上获取 2D 点的 3D 位置,如下图所示:
有一点可以肯定的是那个点的Z是0,但是如何得到X和那个点的Y.
由于你的情况仅限于平原,简单的方法是使用Homography。
首先 undistort your image. Then use findHomography 计算将像素坐标(图像)转换为实际坐标(欧氏 space 例如以厘米为单位)的单应矩阵。与此类似的内容:
#include <opencv2/calib3d.hpp>
//...
//points on undistorted image (in pixel). more is better
vector<Point2f> src_points = { Point2f(123,321), Point2f(456,654), Point2f(789,987), Point2f(123,321) };
//points on chessboard (e.g. in cm)
vector<Point2f> dst_points = { Point2f(0, 0), Point2f(12.5, 0), Point2f(0, 16.5), Point2f(12.5, 16.5) };
Mat H = findHomography(src_points, dst_points, RANSAC);
//print euclidean coordinate of new point on undistorted image (in pixel)
cout << H * Mat(Point3d(125, 521, 0)) << endl;
您可以通过 3 个简单的步骤解决此问题:
第 1 步:
通过反转相机投影模型计算对应于给定 2d 图像点的光线的 3d 方向矢量,以相机的坐标系表示:
std::vector<cv::Point2f> imgPt = {{u,v}}; // Input image point
std::vector<cv::Point2f> normPt;
cv::undistortPoints (imgPt, normPt, cameraMatrix, distCoeffs);
cv::Matx31f ray_dir_cam(normPt[0].x, normPt[0].y, 1);
// 'ray_dir_cam' is the 3d direction of the ray in camera coordinate frame
// In camera coordinate frame, this ray originates from the camera center at (0,0,0)
第 2 步:
使用相机和棋盘之间的相对位姿,在附加到棋盘的坐标系中计算此射线矢量的 3d 方向:
// solvePnP typically gives you 'rvec_cam_chessboard' and 'tvec_cam_chessboard'
// Inverse this pose to get the pose mapping camera coordinates to chessboard coordinates
cv::Matx33f R_cam_chessboard;
cv::Rodrigues(rvec_cam_chessboard, R_cam_chessboard);
cv::Matx33f R_chessboard_cam = R_cam_chessboard.t();
cv::Matx31f t_cam_chessboard = tvec_cam_chessboard;
cv::Matx31f pos_cam_wrt_chessboard = -R_chessboard_cam*t_cam_chessboard;
// Map the ray direction vector from camera coordinates to chessboard coordinates
cv::Matx31f ray_dir_chessboard = R_chessboard_cam * ray_dir_cam;
第 3 步:
通过计算 3d 射线与 Z=0 的棋盘平面之间的交点来找到所需的 3d 点:
// Expressed in the coordinate frame of the chessboard, the ray originates from the
// 3d position of the camera center, i.e. 'pos_cam_wrt_chessboard', and its 3d
// direction vector is 'ray_dir_chessboard'
// Any point on this ray can be expressed parametrically using its depth 'd':
// P(d) = pos_cam_wrt_chessboard + d * ray_dir_chessboard
// To find the intersection between the ray and the plane of the chessboard, we
// compute the depth 'd' for which the Z coordinate of P(d) is equal to zero
float d_intersection = -pos_cam_wrt_chessboard.val[2]/ray_dir_chessboard.val[2];
cv::Matx31f intersection_point = pos_cam_wrt_chessboard + d_intersection * ray_dir_chessboard;
我有一个已知 cameraMatrix
和 distCoeffs
的固定相机。我还有一个固定的棋盘,transform
和 rotation
向量也是使用 solvePnP
计算的。
我想知道如何在棋盘所在的同一平面上获取 2D 点的 3D 位置,如下图所示:
有一点可以肯定的是那个点的Z是0,但是如何得到X和那个点的Y.
由于你的情况仅限于平原,简单的方法是使用Homography。
首先 undistort your image. Then use findHomography 计算将像素坐标(图像)转换为实际坐标(欧氏 space 例如以厘米为单位)的单应矩阵。与此类似的内容:
#include <opencv2/calib3d.hpp>
//...
//points on undistorted image (in pixel). more is better
vector<Point2f> src_points = { Point2f(123,321), Point2f(456,654), Point2f(789,987), Point2f(123,321) };
//points on chessboard (e.g. in cm)
vector<Point2f> dst_points = { Point2f(0, 0), Point2f(12.5, 0), Point2f(0, 16.5), Point2f(12.5, 16.5) };
Mat H = findHomography(src_points, dst_points, RANSAC);
//print euclidean coordinate of new point on undistorted image (in pixel)
cout << H * Mat(Point3d(125, 521, 0)) << endl;
您可以通过 3 个简单的步骤解决此问题:
第 1 步:
通过反转相机投影模型计算对应于给定 2d 图像点的光线的 3d 方向矢量,以相机的坐标系表示:
std::vector<cv::Point2f> imgPt = {{u,v}}; // Input image point
std::vector<cv::Point2f> normPt;
cv::undistortPoints (imgPt, normPt, cameraMatrix, distCoeffs);
cv::Matx31f ray_dir_cam(normPt[0].x, normPt[0].y, 1);
// 'ray_dir_cam' is the 3d direction of the ray in camera coordinate frame
// In camera coordinate frame, this ray originates from the camera center at (0,0,0)
第 2 步:
使用相机和棋盘之间的相对位姿,在附加到棋盘的坐标系中计算此射线矢量的 3d 方向:
// solvePnP typically gives you 'rvec_cam_chessboard' and 'tvec_cam_chessboard'
// Inverse this pose to get the pose mapping camera coordinates to chessboard coordinates
cv::Matx33f R_cam_chessboard;
cv::Rodrigues(rvec_cam_chessboard, R_cam_chessboard);
cv::Matx33f R_chessboard_cam = R_cam_chessboard.t();
cv::Matx31f t_cam_chessboard = tvec_cam_chessboard;
cv::Matx31f pos_cam_wrt_chessboard = -R_chessboard_cam*t_cam_chessboard;
// Map the ray direction vector from camera coordinates to chessboard coordinates
cv::Matx31f ray_dir_chessboard = R_chessboard_cam * ray_dir_cam;
第 3 步:
通过计算 3d 射线与 Z=0 的棋盘平面之间的交点来找到所需的 3d 点:
// Expressed in the coordinate frame of the chessboard, the ray originates from the
// 3d position of the camera center, i.e. 'pos_cam_wrt_chessboard', and its 3d
// direction vector is 'ray_dir_chessboard'
// Any point on this ray can be expressed parametrically using its depth 'd':
// P(d) = pos_cam_wrt_chessboard + d * ray_dir_chessboard
// To find the intersection between the ray and the plane of the chessboard, we
// compute the depth 'd' for which the Z coordinate of P(d) is equal to zero
float d_intersection = -pos_cam_wrt_chessboard.val[2]/ray_dir_chessboard.val[2];
cv::Matx31f intersection_point = pos_cam_wrt_chessboard + d_intersection * ray_dir_chessboard;