如何使用 Farneback 的密集光流方法跟踪稀疏特征？

Question

我已经可以使用在 OpenCV 中实现的 Lucas Kanade 跟踪 SIFT 或 SURF 特征，因为 Lucas Kanade 以任何方式跟踪稀疏特征，但是我正在尝试使用在 OpenCV 中实现的 Farneback 光流算法来跟踪那些稀疏特征，是否存在一个算法？

Realtime Dense Optical flow tracking

请查看此视频：

开发人员声称他们使用密集方法 "Farneback" 而不是稀疏方法 "Lucas-Kanade" 来跟踪选定的稀疏特征。他们是怎么做到的？

Answer 1

OpenCV 中有一个函数 calcOpticalFlowFarneback() 可以做到这一点。

Answer 2

要跟踪具有密集光流场的特征 flow 可以按如下方式完成：

// read images    
cv:Mat prevImg = cv::imread( filename0 ); // image data at time t
cv::Mat currImg = cv::imread( filename1 ); // image data at time t and t + 1
cv::Mat flowMat; // storage for dese optical flow field
std::vector<cv::Point2f> prevPoints; // points to be track

// initialize points to track (example)
prevPoints.push_back( cv::Point2f( 50.3f, 30.f ) );
std::vector<cv::Point2f> currPoints( prevPoints.size()); // tracked point position

 // compute dense flow field (example)
cv::calcOpticalFlowFarneback(prevImg, currImg, flowMat, 0.4, 1, 12, 2, 8, 1.2, 0);

// track points based on dense optical flow field and bilinear interpolation
for( unsigned int n = 0; n < prevPoints.size(); ++n )
{
  float ix = floor(prevPoints[n].x);
  float iy = floor(prevPoints[n].y);
  float wx = prevPoints[n].x - ix;
  float wy = prevPoints[n].y - iy;
  float w00 = (1.f - wx) * (1.f - wy);
  float w10 = (1.f - wx) * wy;
  float w01 = wx * (1.f - wy);
  float w11 = wx * wy;
  if( prevPoints[n].x >= flowMat.cols - 1 || prevPoints[n].y >= flowMat.rows - 1)
  {
    // these points are out of the image roi and cannot be tracked.
    currPoints[n] = prevPoints[n];
  }
  else
  {
   /*
   bilinear interpolation of the flow vector from the flow field at a given location.
   The bilinear interpolation has to be applied since the points to track can be given at subpixel level 
   */
   currPoints[n] = prevPoints[n] 
                 + flowMat.at<cv::Point2f>(iy, ix) * w00 
                 + flowMat.at<cv::Point2f>(iy+1, ix) * w10 
                 + flowMat.at<cv::Point2f>(iy, ix+1) * w01 
                 + flowMat.at<cv::Point2f>(iy+1, ix+1) * w11;
  }
}

}

如何使用 Farneback 的密集光流方法跟踪稀疏特征？

How can I track sparse features using a dense optical flow approach like Farneback's?

c++

opencv

opticalflow

computer-vision