如何在 Dlib C++ 中获取头部姿势估计的 3D 坐标轴

Question

Dlib C++ 可以很好地检测地标和估计面部姿势。但是，如何获得头部姿势的 3D 坐标轴方向 (x,y,z)？

Answer 1

我也遇到了同样的问题，前一段时间，搜索并找到了 1-2 篇有用的博客文章，这个 link would get you an overview of the techniques involved, If you only need to calculate the 3D pose in decimal places then you may skip the OpenGL rendering part, However if you want to visually get the Feedback then you may try with OpenGL as well, But I would suggest you to ignore the OpenGL part as a beginner, So the smallest working code snippet extracted from github 页面看起来像这样：

// Reading image using OpenCV, you may use dlib as well.
cv::Mat img = cv::imread(imagePath);

std::vector<double> rv(3), tv(3);
cv::Mat rvec(rv),tvec(tv);
cv::Vec3d eav;

// Labelling the 3D Points derived from a 3D model of human face.
// You may replace these points as per your custom 3D head model if any
std::vector<cv::Point3f > modelPoints;
modelPoints.push_back(cv::Point3f(2.37427,110.322,21.7776));    // l eye (v 314)
modelPoints.push_back(cv::Point3f(70.0602,109.898,20.8234));    // r eye (v 0)
modelPoints.push_back(cv::Point3f(36.8301,78.3185,52.0345));    //nose (v 1879)
modelPoints.push_back(cv::Point3f(14.8498,51.0115,30.2378));    // l mouth (v 1502)
modelPoints.push_back(cv::Point3f(58.1825,51.0115,29.6224));    // r mouth (v 695)
modelPoints.push_back(cv::Point3f(-61.8886f,127.797,-89.4523f));  // l ear (v 2011)
modelPoints.push_back(cv::Point3f(127.603,126.9,-83.9129f));     // r ear (v 1138)

// labelling the position of corresponding feature points on the input image.
std::vector<cv::Point2f> srcImagePoints = {cv::Point2f(442, 442), // left eye
                                           cv::Point2f(529, 426), // right eye
                                           cv::Point2f(501, 479), // nose
                                           cv::Point2f(469, 534), //left lip corner
                                           cv::Point2f(538, 521), // right lip corner
                                           cv::Point2f(349, 457), // left ear
                                           cv::Point2f(578, 415) // right ear};


cv::Mat ip(srcImagePoints);

cv::Mat op = cv::Mat(modelPoints);
cv::Scalar m = mean(cv::Mat(modelPoints));

rvec = cv::Mat(rv);
double _d[9] = {1,0,0,
                0,-1,0,
                0,0,-1};
Rodrigues(cv::Mat(3,3,CV_64FC1,_d),rvec);
tv[0]=0;tv[1]=0;tv[2]=1;
tvec = cv::Mat(tv);


double max_d = MAX(img.rows,img.cols);
double _cm[9] = {max_d,     0, (double)img.cols/2.0,
                 0    , max_d, (double)img.rows/2.0,
                 0    ,     0,                  1.0};
cv::Mat camMatrix = cv::Mat(3,3,CV_64FC1, _cm);

double _dc[] = {0,0,0,0};
solvePnP(op,ip,camMatrix,cv::Mat(1,4,CV_64FC1,_dc),rvec,tvec,false,CV_EPNP);

double rot[9] = {0};
cv::Mat rotM(3,3,CV_64FC1,rot);
Rodrigues(rvec,rotM);
double* _r = rotM.ptr<double>();
printf("rotation mat: \n %.3f %.3f %.3f\n%.3f %.3f %.3f\n%.3f %.3f %.3f\n",
       _r[0],_r[1],_r[2],_r[3],_r[4],_r[5],_r[6],_r[7],_r[8]);

printf("trans vec: \n %.3f %.3f %.3f\n",tv[0],tv[1],tv[2]);

double _pm[12] = {_r[0],_r[1],_r[2],tv[0],
                  _r[3],_r[4],_r[5],tv[1],
                  _r[6],_r[7],_r[8],tv[2]};

cv::Mat tmp,tmp1,tmp2,tmp3,tmp4,tmp5;
cv::decomposeProjectionMatrix(cv::Mat(3,4,CV_64FC1,_pm),tmp,tmp1,tmp2,tmp3,tmp4,tmp5,eav);
printf("Face Rotation Angle:  %.5f %.5f %.5f\n",eav[0],eav[1],eav[2]);

输出：

                       **X**     **Y**    **Z**

Face Rotation Angle:  171.44027 -8.72583 -9.90596

如何在 Dlib C++ 中获取头部姿势估计的 3D 坐标轴

How to get 3D coordinate Axes of head pose estimation in Dlib C++

opencv

face-recognition

head

coordinate

dlib