计算具有共享地平面的 2 个单应平面之间的距离

Question

我觉得最简单的就是用图片来解释问题：

我有两个立方体（大小相同）放在 table 上。他们的一侧标有绿色（便于追踪）。我想以立方体大小为单位计算左立方体与右立方体（图中红线）的相对位置。

这可能吗？我知道如果这两个绿色面有共同的平面，问题会很简单——就像立方体的顶面，但我不能用它来跟踪。我只会计算一个正方形的单应性并与其他立方体角相乘。

我是否应该 'rotate' 单应矩阵乘以 90 度旋转矩阵得到 'ground' 单应矩阵？我计划在智能手机场景中进行处理，因此陀螺仪、相机固有参数可能具有任何值。

Answer 1

这是可能的。让我们假设（或声明）table 是 z=0 平面，并且您的第一个框位于该平面的原点。这意味着左框的绿色角具有 (table-) 坐标 (0,0,0),(1,0,0),(0,0,1) 和 (1,0,1 ). （您的盒子尺寸为 1）。您还拥有这些点的像素坐标。如果您将这些 2d 和 3d 值（以及相机的固有特性和失真）赋予 cv::solvePnP，您将获得相机相对于您的盒子（和平面）的姿势。

在下一步中，您必须使 table 平面与从相机中心穿过第二个绿色框右下角像素的光线相交。这个交叉点看起来像 (x,y,0) 并且 [x-1,y] 将在您的框的右角之间平移。

Answer 2

如果您拥有所有信息（相机内在信息），您可以按照 FooBar 回答的方式进行操作。

但是您可以使用单应性更直接地使用点位于平面上的信息（无需计算射线等）：

计算图像平面和地平面之间的单应性。不幸的是，您需要 4 个点对应，但图像中只有 3 个可见的立方体点接触地平面。相反，您可以使用立方体的顶平面，可以测量相同的距离。

首先是代码：

int main()
{
    // calibrate plane distance for boxes
    cv::Mat input = cv::imread("../inputData/BoxPlane.jpg");


    // if we had 4 known points on the ground plane, we could use the ground plane but here we instead use the top plane
    // points on real world plane: height = 1: // so it's not measured on the ground plane but on the "top plane" of the cube
    std::vector<cv::Point2f> objectPoints;  
    objectPoints.push_back(cv::Point2f(0,0)); // top front
    objectPoints.push_back(cv::Point2f(1,0)); // top right
    objectPoints.push_back(cv::Point2f(0,1)); // top left
    objectPoints.push_back(cv::Point2f(1,1)); // top back

    // image points:
    std::vector<cv::Point2f> imagePoints;
    imagePoints.push_back(cv::Point2f(141,302));// top front
    imagePoints.push_back(cv::Point2f(334,232));// top right
    imagePoints.push_back(cv::Point2f(42,231)); // top left
    imagePoints.push_back(cv::Point2f(223,177));// top back

    cv::Point2f pointToMeasureInImage(741,200); // bottom right of second box


    // for transform we need the point(s) to be in a vector
    std::vector<cv::Point2f> sourcePoints;
    sourcePoints.push_back(pointToMeasureInImage);
    //sourcePoints.push_back(pointToMeasureInImage);
    sourcePoints.push_back(cv::Point2f(718,141));
    sourcePoints.push_back(imagePoints[0]);


    // list with points that correspond to sourcePoints. This is not needed but used to create some ouput
    std::vector<int> distMeasureIndices;
    distMeasureIndices.push_back(1);
    //distMeasureIndices.push_back(0);
    distMeasureIndices.push_back(3);
    distMeasureIndices.push_back(2);


    // draw points for visualization
    for(unsigned int i=0; i<imagePoints.size(); ++i)
    {
        cv::circle(input, imagePoints[i], 5, cv::Scalar(0,255,255));
    }
    //cv::circle(input, pointToMeasureInImage, 5, cv::Scalar(0,255,255));
    //cv::line(input, imagePoints[1], pointToMeasureInImage, cv::Scalar(0,255,255), 2);

    // compute the relation between the image plane and the real world top plane of the cubes
    cv::Mat homography = cv::findHomography(imagePoints, objectPoints);



    std::vector<cv::Point2f> destinationPoints;
    cv::perspectiveTransform(sourcePoints, destinationPoints, homography);

    // compute the distance between some defined points (here I use the input points but could be something else)
    for(unsigned int i=0; i<sourcePoints.size(); ++i)
    {
        std::cout << "distance: " << cv::norm(destinationPoints[i] - objectPoints[distMeasureIndices[i]]) << std::endl; 

        cv::circle(input, sourcePoints[i], 5, cv::Scalar(0,255,255));
        // draw the line which was measured
        cv::line(input, imagePoints[distMeasureIndices[i]], sourcePoints[i], cv::Scalar(0,255,255), 2);
    }


    // just for fun, measure distances on the 2nd box:
    float distOn2ndBox = cv::norm(destinationPoints[0]-destinationPoints[1]);
    std::cout << "distance on 2nd box: " << distOn2ndBox << " which should be near 1.0" << std::endl;
    cv::line(input, sourcePoints[0], sourcePoints[1], cv::Scalar(255,0,255), 2);


    cv::imshow("input", input);
    cv::waitKey(0);
    return 0;
}

这是我要解释的输出：

distance: 2.04674
distance: 2.82184
distance: 1
distance on 2nd box: 0.882265 which should be near 1.0

这些距离是：

1. the yellow bottom one from one box to the other
2. the yellow top one
3. the yellow one on the first box
4. the pink one

所以红线（你要求的）的长度应该几乎正好是立方体边长的 2 倍。但是如您所见，我们有一些错误。

better/more 在单应性计算之前更正您的像素位置，您的结果越准确。

你需要一个针孔相机模型，所以不扭曲你的相机（在实际应用中）。

还要记住，如果在那里有 4 个可见的线性点（不在同一条线上），您可以计算地平面上的距离！

计算具有共享地平面的 2 个单应平面之间的距离

Calculating distance between 2 homography planes that have shared ground plane

opencv

image-processing

distance

homography