如何使用相机矩阵以毫米为单位找到图像中某个点的位置？
How to find the location of a point in an image in millimeters using camera matrix?
我使用的是标准 640x480 网络摄像头。我在 Python 3 的 OpenCV 中完成了相机校准。这是我正在使用的代码。该代码正在运行，并成功地为我提供了 Camera Matrix 和 Distortion Coefficients。
现在，如何找到 我的场景图像中 640 像素 中有多少毫米。我将网络摄像头水平安装在 table 上方，在 table 上放置了一个机械臂。我正在使用相机找到对象的 centroid。使用 Camera Matrix 我的目标是将该对象的位置（例如 300x200 像素）转换为毫米单位，以便我可以将毫米数提供给机械臂来拾取该对象。
我已经搜索但没有找到任何相关信息。
请告诉我是否有任何方程式或方法。非常感谢！
import numpy as np
import cv2
import yaml
import os
# Parameters
#TODO : Read from file
n_row=4 #Checkerboard Rows
n_col=6 #Checkerboard Columns
n_min_img = 10 # number of images needed for calibration
square_size = 40 # size of each individual box on Checkerboard in mm
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001) # termination criteria
corner_accuracy = (11,11)
result_file = "./calibration.yaml" # Output file having camera matrix
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(n_row-1,n_col-1,0)
objp = np.zeros((n_row*n_col,3), np.float32)
objp[:,:2] = np.mgrid[0:n_row,0:n_col].T.reshape(-1,2) * square_size
# Intialize camera and window
camera = cv2.VideoCapture(0) #Supposed to be the only camera
if not camera.isOpened():
print("Camera not found!")
quit()
width = int(camera.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(camera.get(cv2.CAP_PROP_FRAME_HEIGHT))
cv2.namedWindow("Calibration")
# Usage
def usage():
print("Press on displayed window : \n")
print("[space] : take picture")
print("[c] : compute calibration")
print("[r] : reset program")
print("[ESC] : quit")
usage()
Initialization = True
while True:
if Initialization:
print("Initialize data structures ..")
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
n_img = 0
Initialization = False
tot_error=0
# Read from camera and display on windows
ret, img = camera.read()
cv2.imshow("Calibration", img)
if not ret:
print("Cannot read camera frame, exit from program!")
camera.release()
cv2.destroyAllWindows()
break
# Wait for instruction
k = cv2.waitKey(50)
# SPACE pressed to take picture
if k%256 == 32:
print("Adding image for calibration...")
imgGray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(imgGray, (n_row,n_col),None)
# If found, add object points, image points (after refining them)
if not ret:
print("Cannot found Chessboard corners!")
else:
print("Chessboard corners successfully found.")
objpoints.append(objp)
n_img +=1
corners2 = cv2.cornerSubPix(imgGray,corners,corner_accuracy,(-1,-1),criteria)
imgpoints.append(corners2)
# Draw and display the corners
imgAugmnt = cv2.drawChessboardCorners(img, (n_row,n_col), corners2,ret)
cv2.imshow('Calibration',imgAugmnt)
cv2.waitKey(500)
# "c" pressed to compute calibration
elif k%256 == 99:
if n_img <= n_min_img:
print("Only ", n_img , " captured, ", " at least ", n_min_img , " images are needed")
else:
print("Computing calibration ...")
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, (width,height),None,None)
if not ret:
print("Cannot compute calibration!")
else:
print("Camera calibration successfully computed")
# Compute reprojection errors
for i in range(len(objpoints)):
imgpoints2, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
error = cv2.norm(imgpoints[i],imgpoints2, cv2.NORM_L2)/len(imgpoints2)
tot_error += error
print("Camera matrix: ", mtx)
print("Distortion coeffs: ", dist)
print("Total error: ", tot_error)
print("Mean error: ", np.mean(error))
# Saving calibration matrix
try:
os.remove(result_file) #Delete old file first
except Exception as e:
#print(e)
pass
print("Saving camera matrix .. in ",result_file)
data={"camera_matrix": mtx.tolist(), "dist_coeff": dist.tolist()}
with open(result_file, "w") as f:
yaml.dump(data, f, default_flow_style=False)
# ESC pressed to quit
elif k%256 == 27:
print("Escape hit, closing...")
camera.release()
cv2.destroyAllWindows()
break
# "r" pressed to reset
elif k%256 ==114:
print("Reset program...")
Initialization = True
这是相机矩阵：
818.6 0 324.4
0 819.1 237.9
0 0 1
失真系数：
0.34 -5.7 0 0 33.45
再见，
其实我在想，你应该能够以一种简单的方式解决你的问题：
mm_per_pixel = real_mm_width : 640px
假设相机最初与要拾取的对象的计划平行移动[即fixed distance]，real_mm_width 可以找到测量与图片的那些 640 像素对应的物理距离。举个例子，假设你发现 real_mm_width = 32cm = 320mm，所以你得到 mm_per_pixel = 0.5mm/px。对于固定距离，此比率不会改变
看来也是official documentation的建议：
This consideration helps us to find only X,Y values. Now for X,Y
values, we can simply pass the points as (0,0), (1,0), (2,0), ...
which denotes the location of points. In this case, the results we get
will be in the scale of size of chess board square. But if we know the
square size, (say 30 mm), we can pass the values as (0,0), (30,0),
(60,0), ... . Thus, we get the results in mm
然后你只需要将质心坐标转换为像素[例如(pixel_x_centroid, pixel_y_centroid) = (300px, 200px)] 到 mm 使用：
mm_x_centroid = pixel_x_centroid * mm_per_pixel
mm_y_centroid = pixel_y_centroid * mm_per_pixel
这会给你最终答案：
(mm_x_centroid, mm_y_centroid) = (150mm, 100mm)
看到相同事物的另一种方式是这个比例，其中第一个成员是 measurable/known 比例：
real_mm_width : 640px = mm_x_centroid : pixel_x_centroid = mm_y_centroid = pixel_y_centroid
祝你有美好的一天，
安东尼奥
我使用的是标准 640x480 网络摄像头。我在 Python 3 的 OpenCV 中完成了相机校准。这是我正在使用的代码。该代码正在运行，并成功地为我提供了 Camera Matrix 和 Distortion Coefficients。 现在，如何找到 我的场景图像中 640 像素 中有多少毫米。我将网络摄像头水平安装在 table 上方，在 table 上放置了一个机械臂。我正在使用相机找到对象的 centroid。使用 Camera Matrix 我的目标是将该对象的位置（例如 300x200 像素）转换为毫米单位，以便我可以将毫米数提供给机械臂来拾取该对象。 我已经搜索但没有找到任何相关信息。 请告诉我是否有任何方程式或方法。非常感谢！
import numpy as np
import cv2
import yaml
import os
# Parameters
#TODO : Read from file
n_row=4 #Checkerboard Rows
n_col=6 #Checkerboard Columns
n_min_img = 10 # number of images needed for calibration
square_size = 40 # size of each individual box on Checkerboard in mm
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001) # termination criteria
corner_accuracy = (11,11)
result_file = "./calibration.yaml" # Output file having camera matrix
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(n_row-1,n_col-1,0)
objp = np.zeros((n_row*n_col,3), np.float32)
objp[:,:2] = np.mgrid[0:n_row,0:n_col].T.reshape(-1,2) * square_size
# Intialize camera and window
camera = cv2.VideoCapture(0) #Supposed to be the only camera
if not camera.isOpened():
print("Camera not found!")
quit()
width = int(camera.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(camera.get(cv2.CAP_PROP_FRAME_HEIGHT))
cv2.namedWindow("Calibration")
# Usage
def usage():
print("Press on displayed window : \n")
print("[space] : take picture")
print("[c] : compute calibration")
print("[r] : reset program")
print("[ESC] : quit")
usage()
Initialization = True
while True:
if Initialization:
print("Initialize data structures ..")
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
n_img = 0
Initialization = False
tot_error=0
# Read from camera and display on windows
ret, img = camera.read()
cv2.imshow("Calibration", img)
if not ret:
print("Cannot read camera frame, exit from program!")
camera.release()
cv2.destroyAllWindows()
break
# Wait for instruction
k = cv2.waitKey(50)
# SPACE pressed to take picture
if k%256 == 32:
print("Adding image for calibration...")
imgGray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(imgGray, (n_row,n_col),None)
# If found, add object points, image points (after refining them)
if not ret:
print("Cannot found Chessboard corners!")
else:
print("Chessboard corners successfully found.")
objpoints.append(objp)
n_img +=1
corners2 = cv2.cornerSubPix(imgGray,corners,corner_accuracy,(-1,-1),criteria)
imgpoints.append(corners2)
# Draw and display the corners
imgAugmnt = cv2.drawChessboardCorners(img, (n_row,n_col), corners2,ret)
cv2.imshow('Calibration',imgAugmnt)
cv2.waitKey(500)
# "c" pressed to compute calibration
elif k%256 == 99:
if n_img <= n_min_img:
print("Only ", n_img , " captured, ", " at least ", n_min_img , " images are needed")
else:
print("Computing calibration ...")
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, (width,height),None,None)
if not ret:
print("Cannot compute calibration!")
else:
print("Camera calibration successfully computed")
# Compute reprojection errors
for i in range(len(objpoints)):
imgpoints2, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
error = cv2.norm(imgpoints[i],imgpoints2, cv2.NORM_L2)/len(imgpoints2)
tot_error += error
print("Camera matrix: ", mtx)
print("Distortion coeffs: ", dist)
print("Total error: ", tot_error)
print("Mean error: ", np.mean(error))
# Saving calibration matrix
try:
os.remove(result_file) #Delete old file first
except Exception as e:
#print(e)
pass
print("Saving camera matrix .. in ",result_file)
data={"camera_matrix": mtx.tolist(), "dist_coeff": dist.tolist()}
with open(result_file, "w") as f:
yaml.dump(data, f, default_flow_style=False)
# ESC pressed to quit
elif k%256 == 27:
print("Escape hit, closing...")
camera.release()
cv2.destroyAllWindows()
break
# "r" pressed to reset
elif k%256 ==114:
print("Reset program...")
Initialization = True
这是相机矩阵：
818.6 0 324.4
0 819.1 237.9
0 0 1
失真系数：
0.34 -5.7 0 0 33.45
再见，
其实我在想，你应该能够以一种简单的方式解决你的问题：
mm_per_pixel = real_mm_width : 640px
假设相机最初与要拾取的对象的计划平行移动[即fixed distance]，real_mm_width 可以找到测量与图片的那些 640 像素对应的物理距离。举个例子，假设你发现 real_mm_width = 32cm = 320mm，所以你得到 mm_per_pixel = 0.5mm/px。对于固定距离，此比率不会改变
看来也是official documentation的建议：
This consideration helps us to find only X,Y values. Now for X,Y values, we can simply pass the points as (0,0), (1,0), (2,0), ... which denotes the location of points. In this case, the results we get will be in the scale of size of chess board square. But if we know the square size, (say 30 mm), we can pass the values as (0,0), (30,0), (60,0), ... . Thus, we get the results in mm
然后你只需要将质心坐标转换为像素[例如(pixel_x_centroid, pixel_y_centroid) = (300px, 200px)] 到 mm 使用：
mm_x_centroid = pixel_x_centroid * mm_per_pixel
mm_y_centroid = pixel_y_centroid * mm_per_pixel
这会给你最终答案：
(mm_x_centroid, mm_y_centroid) = (150mm, 100mm)
看到相同事物的另一种方式是这个比例，其中第一个成员是 measurable/known 比例：
real_mm_width : 640px = mm_x_centroid : pixel_x_centroid = mm_y_centroid = pixel_y_centroid
祝你有美好的一天，
安东尼奥