我怎样才能使 AR 视频始终面对用户的相机，并带有标记 vuforia native android

Question

我正在使用原生 android vuforia 开发标记 AR 我想做的是根据摄像头移动我的视频对象（它应该始终面向摄像头）我尝试跟随但它不起作用但有时它的工作，但它不是持久性

public void renderFrame(State state, float[] projectionMatrix) {

        mSampleAppRenderer.renderVideoBackground();
        GLES20.glEnable(GLES20.GL_DEPTH_TEST);

        isTracking = false;

        for (int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++) {
            TrackableResult trackableResult = state.getTrackableResult(tIdx);
            ImageTarget imageTarget = (ImageTarget) trackableResult
                    .getTrackable();
            imageTarget.startExtendedTracking();

            isTracking = true;

            float[] modelViewMatrixVideo = Tool.convertPose2GLMatrix(
                    trackableResult.getPose()).getData();
            float[] modelViewProjectionVideo = new float[16];



            Matrix44F invTranspMV = SampleMath.Matrix44FTranspose(SampleMath.Matrix44FInverse(Tool.convertPose2GLMatrix(trackableResult.getPose())));


            Matrix.translateM(modelViewMatrixVideo, 0, 0f, 0f, 1f);
            Matrix.rotateM(modelViewMatrixVideo, 0, (float) Math.toDegrees(Math.asin(-invTranspMV.getData()[6])), 0.0f, 0.f, 1.0f);


            Matrix.multiplyMM(modelViewProjectionVideo, 0,
                    projectionMatrix, 0, modelViewMatrixVideo, 0);


            GLES20.glEnable(GLES20.GL_BLEND);
            GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
            GLES20.glUseProgram(videoPlaybackShaderID);


            GLES20.glVertexAttribPointer(videoPlaybackVertexHandle, 3,
                    GLES20.GL_FLOAT, false, 0, quadVertices);

            GLES20.glVertexAttribPointer(videoPlaybackTexCoordHandle,
                    2, GLES20.GL_FLOAT, false, 0,
                    fillBuffer(videoQuadTextureCoordsTransformedStones));

            GLES20.glEnableVertexAttribArray(videoPlaybackVertexHandle);
            GLES20.glEnableVertexAttribArray(videoPlaybackTexCoordHandle);

            GLES20.glActiveTexture(GLES20.GL_TEXTURE0);


            GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
                    videoPlaybackTextureID);
            GLES20.glUniformMatrix4fv(videoPlaybackMVPMatrixHandle, 1,
                    false, modelViewProjectionVideo, 0);

            GLES20.glDrawElements(GLES20.GL_TRIANGLES, NUM_QUAD_INDEX,
                    GLES20.GL_UNSIGNED_SHORT, quadIndices);

            GLES20.glDisableVertexAttribArray(videoPlaybackVertexHandle);
            GLES20.glDisableVertexAttribArray(videoPlaybackTexCoordHandle);

            GLES20.glUseProgram(0);
            GLES20.glDisable(GLES20.GL_BLEND);

            SampleUtils.checkGLError("VideoPlayback renderFrame");
        }

        GLES20.glDisable(GLES20.GL_DEPTH_TEST);

        Renderer.getInstance().end();


    }

到目前为止，我已经尝试过上面的方法，它有时可以工作，但不能正常工作请帮助我，我正尝试从一周开始这样做。

Answer 1

I am trying to do is to move my video object according to camera (it should always face camera)

如果你想让一个物体面向相机（广告牌），那么你必须使用一个模型矩阵，它是逆视图矩阵，但没有平移部分。使用 Matrix44FInverse, to geht the inverse matrix of a Matrix44F:

public void renderFrame(State state, float[] projectionMatrix) {

    .....

    // get the view matrix and set translation part to (0, 0, 0)
    float[] tempViewMat = Tool.convertPose2GLMatrix(trackableResult.getPose()).getData();
    tempViewMat[12] = 0;
    tempViewMat[13] = 0;
    tempViewMat[14] = 0;

    // create the billboard matrix 
    Matrix44F billboardMatrix = new Matrix44F();
    billboardMatrix.setData(tempViewMat);
    billboardMatrix = SampleMath.Matrix44FInverse(billboardMatrix);

    // calculate the model view projection matrix

    float[] viewMatrixVideo = Tool.convertPose2GLMatrix(trackableResult.getPose()).getData();

    float[] modelViewVideo = new float[16];
    Matrix.multiplyMM(modelViewVideo, 0, viewMatrixVideo, 0, billboardMatrix.getData(), 0);

    float[] modelViewProjectionVideo = new float[16];
    Matrix.multiplyMM(modelViewProjectionVideo, 0, projectionMatrix, 0, modelViewVideo, 0);

    .....
}

I used your code but my AR is rotating in every direction but i want some different thing I have one video that I want to place it vertical on marker and if user move left or right only then i want to rotate it to face camera so that i will see my AR video same that is vertical from any view

你想要做的是固定向上的方向，但将法向量定向到视线。

视线是 Right-Handed Coordinate System 中视图 space 的反 Z 轴。

Matrix44F inverse_view = SampleMath.Matrix44FInverse(
    Tool.convertPose2GLMatrix(trackableResult.getPose()));

// line of sight  
Vec3F los = new Vec3F(-inverse_view[8], -inverse_view[9], -inverse_view[10] );

Y 轴 (0, 1, 0) 或 Z 轴 (0, 1, 0)模型必须是方向矩阵的 Z 轴。 X 轴是视线与方向矩阵的 Z 轴的叉积 (Vec3FCross)。

例如

Vec3F z_axis = new Vec3F(0, 0, 1);
Vec3F x_axis = Vec3FNormalize(Vec3FCross(los, z_axis));
Vec3F y_axis = Vec3FCross(z_axis, x_axis);

float[] orientationMatrix = new float[]{
    x_axis.getData()[0], x_axis.getData()[1], x_axis.getData()[2], 0,
    y_axis.getData()[0], y_axis.getData()[1], y_axis.getData()[2], 0,
    z_axis.getData()[0], z_axis.getData()[1], z_axis.getData()[2], 0,
    0,                   0,                   0,                   1
};

// calculate the model view projection matrix

float[] viewMatrixVideo = Tool.convertPose2GLMatrix(trackableResult.getPose()).getData();

float[] modelViewVideo = new float[16];
Matrix.multiplyMM(modelViewVideo, 0, viewMatrixVideo, 0, orientationMatrix, 0);

float[] modelViewProjectionVideo = new float[16];
Matrix.multiplyMM(modelViewProjectionVideo, 0, projectionMatrix, 0, modelViewVideo, 0);

我怎样才能使 AR 视频始终面对用户的相机，并带有标记 vuforia native android

How can I make AR video to always face user's camera with marker vuforia native android

android

opengl-es

native

augmented-reality

vuforia