将 Tango 3D 点投影到屏幕 Google Project Tango

Question

Ptoject Tango 提供了一个点云，如何获取点云中 3D 点的像素位置（以米为单位）？

我尝试使用投影矩阵，但我得到的值非常小（0.5、1.3 等）而不是 1234,324（以像素为单位）。

我包括我试过的代码

    //Get the current rotation matrix
    Matrix4 projMatrix =  mRenderer.getCurrentCamera().getProjectionMatrix();



    //Get all the points in the pointcloud and store them as 3D points
    FloatBuffer pointsBuffer =  mPointCloudManager.updateAndGetLatestPointCloudRenderBuffer().floatBuffer;
    Vector3[] points3D = new Vector3[pointsBuffer.capacity()/3];

    int j =0;
    for (int i = 0; i < pointsBuffer.capacity() - 3; i = i + 3) {

        points3D[j]= new Vector3(
                pointsBuffer.get(i),
                pointsBuffer.get(i+1),
                pointsBuffer.get(i+2));
        //Log.v("Points3d", "J: "+ j + " X: " +points3D[j].x + "\tY: "+ points3D[j].y +"\tZ: "+ points3D[j].z );
        j++;
    }


    //Get the projection of the points in the screen.
    Vector3[] points2D = new Vector3[points3D.length];
    for(int i =0; i < points3D.length-1;i++)
    {
        Log.v("Points", "X: " +points3D[i].x + "\tY: "+ points3D[i].y +"\tZ: "+ points3D[i].z );
        points2D[i] = points3D[i].multiply(projMatrix);
        Log.v("Points", "pX: " +points2D[i].x + "\tpY: "+ points2D[i].y +"\tpZ: "+ points2D[i].z );
    }

我使用的例子是点云 java 可以在这里找到 https://github.com/googlesamples/tango-examples-java

更新

TangoCameraIntrinsics ccIntrinsics = mTango.getCameraIntrinsics(TangoCameraIntrinsics.TANGO_CAMERA_COLOR);
    double fx = ccIntrinsics.fx;
    double fy = ccIntrinsics.fy;
    double cx = ccIntrinsics.cx;
    double cy = ccIntrinsics.cy;

    double[][] projMatrix = new double[][] {
            {fx, 0 , -cx},
            {0,  fy, -cy},
            {0,  0,    1}
    };

然后计算我使用的投影点

for(int i =0; i < points3D.length-1;i++)
    {

        double[][] point = new double[][] {
                {points3D[i].x},
                {points3D[i].y},
                {points3D[i].z}
        };

        double [][] point2d = CustomMatrix.multiplyByMatrix(projMatrix, point);

        points2D[i] = new Vector2(0,0);
        if(point2d[2][0]!=0)
        {
            Log.v("temp point", "pX: " +point2d[0][0]/point2d[2][0]+" pY: " +point2d[1][0]/point2d[2][0] );
            points2D[i] = new Vector2(point2d[0][0]/point2d[2][0],point2d[1][0]/point2d[2][0]);
        }

    }

但我认为结果仍然不是预期的结果，例如我得到的结果如下：

pX：-175.58042313027244 pY：-92.573740812066

我觉得不对。

更新按照建议使用彩色相机可获得更好的结果，但点数仍然为负 -1127.8086915171814 年：-652.5887102192332

直接乘以-1可以吗？

Answer 1

您必须将 3D 点与 RGB 相机的固有矩阵相乘才能获得像素坐标。 3D 点位于 Depthcamera 的框架中。您可以通过以下方法获取像素坐标：

$\begin{bmatrix} x\ y\ w \end{bmatrix}_{pixel} = K*\begin{bmatrix} X\ Y\ Z \end{bmatrix}_{3dpoint}$

和

$x = x/w \ y = y/w$

x和y是像素坐标。并且 K 是使用 intrinsics 函数

的参数构造的

$\begin{bmatrix} f_x & 0 & c_x\ 0& f_y & c_y\ 0& 0 & 1 \end{bmatrix}$

将 Tango 3D 点投影到屏幕 Google Project Tango

projecting Tango 3D point to screen Google Project Tango

java

projection

projection-matrix

google-project-tango