OpenCV + Python：计算立体重投影误差

Question

我想做类似于 this question 的事情，但是 stereoCalibrate() 而不是 calibrateCamera()。也就是说，计算立体相机校准的重投影误差。

我的简化示例如下所示：

import cv2
import numpy as np

def calibrate_stereo(w, h, objpoints, imgpoints_l, imgpoints_r):
    stereocalib_criteria = (cv2.TERM_CRITERIA_COUNT + cv2.TERM_CRITERIA_EPS , 1000, 1e-6)
    retval, A1, D1, A2, D2, R, T, E, F = cv2.stereoCalibrate(objpoints,imgpoints_l, imgpoints_r,None,None,None,None, (w,h), flags=0, criteria=stereocalib_criteria)

    return (retval, (A1,D1,A2,D2, R, T, E, F))

def calc_rms_stereo(objectpoints, imgpoints_l, imgpoints_r, A1, D1, A2, D2, R, T):
    tot_error = 0
    total_points = 0

    for i, objpoints in enumerate(objectpoints):
        # calculate world <-> cam1 transformation
        _, rvec_l, tvec_l,_ = cv2.solvePnPRansac(objpoints, imgpoints_l[i], A1, D1)

        # compute reprojection error for cam1
        rp_l, _ = cv2.projectPoints(objpoints, rvec_l, tvec_l, A1, D1)
        tot_error += np.sum(np.square(np.float64(imgpoints_l[i] - rp_l)))
        total_points += len(objpoints)

        # calculate world <-> cam2 transformation
        rvec_r, tvec_r  = cv2.composeRT(rvec_l,tvec_l,cv2.Rodrigues(R)[0],T)[:2]

        # compute reprojection error for cam2
        rp_r,_ = cv2.projectPoints(objpoints, rvec_r, tvec_r, A2, D2)
        tot_error += np.square(imgpoints_r[i] - rp_r).sum()
        total_points += len(objpoints)

    mean_error = np.sqrt(tot_error/total_points)

    return mean_error


if __name__ == "__main__":    
    # omitted: reading values for w,h, objectPoints, imgpoints_l, imgpoints_r from file (format as expected by the OpenCV functions)
    # [...]

    rms, (A1,D1,A2,D2,R,T,_,_) = calibrate_stereo(w, h, objectpoints, imgpoints_l, imgpoints_r)

    print("RMS (stereo calib): {}".format(rms))

    rms_2 = calc_rms_stereo(objectpoints, imgpoints_l, imgpoints_r, A1, D1, A2, D2, R, T)    
    print("RMS (custom calculation):", rms_2)

示例输出：

RMS (stereo calib): 0.14342257926694932
RMS (custom calculation): 0.356273345751

据我所知，stereoCalibrate()源代码中的计算与我的非常相似。我错过了什么？

Ubuntu

上的 OpenCV 3.3.0

Answer 1

我在基于OpenCV实现的自定义立体校准算法后解决了这个问题。

cv2.stereoCalibrate() 内部计算的重投影误差与我的自定义计算之间的差异源于外部参数 rvec_l 和 tvec_l 的不同值。这些向量描述了左摄像头与每个图像的校准图案之间的旋转和平移。 cv2.solvePnpRansac() 仅根据左图的重投影误差生成优化值，而在 cv2.stereoCalibrate() 中，这些值与 R 和 T 一起根据两者的重投影误差进行了优化每个立体对的图像。

如果想精确复制由cv2.stereoCalibrate()编辑的return的RMS值，必须修改cv::stereoCalibrate()的C/C++源代码以return 也优化了外部参数（cv::calibrateCamera() 已经为单眼校准做到了）。

OpenCV + Python：计算立体重投影误差

OpenCV + Python: Calculate stereo reprojection error

python

opencv

camera-calibration

cv2