OpenCV 中 Lucas-Kanade 跟踪器的错误和状态输出
Error and status outputs in Lucas-Kanade tracker in OpenCV
看了OpenCV的文档，我还是不能完全弄清楚st, err在p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)里是什么，什么时候需要用到。
如果你对他们不了解的地方再具体一点就更好了。 documentation for calcOpticalFlowPyrLK() 表示：
status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set in flags parameter; if the flow wasn’t found then the error is not defined (use the status parameter to find such cases).
假设您的图像或帧的边界上有一个特征，而在下一个图像或帧中，该特征消失了。在下一帧中找不到该特征，因此 status 中该特征的相应索引将为 0。否则，status 在该索引处将为 1。
err 值在 status 为 0 的地方没有定义。否则，错误基本上是 "how well do the neighboring pixels around the feature correspond between the two images/frames"。这在 flags 参数下更清楚一点：
OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (see minEigThreshold description); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.
L1距离就是window内的绝对差之和。所以这只是在 images/frames 中的特征周围查找 window 并计算每个 window 中相应像素之间的绝对差，然后将它们相加。所以这就是当标志设置为 not 时会发生的情况。如果是，文档指导我们看minEigThreshold参数的说明：
minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.
因此，使用标志，您可以看到每个特征的最小特征值——如果您想找出要使用的阈值，这很有用。在 OpenCV 文档 (Bouguet00) 中引用的 LK 特征跟踪器的金字塔实现的 original paper 中，它在第 3 节（特征选择）中详细说明了这意味着什么：
So far, we have described the tracking procedure that takes care of following a point u on an image I to another location v on another image J. However, we have not described means to select the point u on I in the first place. This step is called feature selection. It is very intuitive to approach the problem of feature selection once the mathematical ground for tracking is led out. Indeed, the central step of tracking is the computation of the optical flow vector η^k (see pseudo-code of algorithm in section 2.4). At that step, the G matrix is required to be invertible, or in other words, the minimum eigenvalue of G must be large enough (larger than a threshold). This characterizes pixels that are "easy to track".
看了OpenCV的文档，我还是不能完全弄清楚st, err在p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)里是什么，什么时候需要用到。
如果你对他们不了解的地方再具体一点就更好了。 documentation for calcOpticalFlowPyrLK() 表示：
status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set inflagsparameter; if the flow wasn’t found then the error is not defined (use thestatusparameter to find such cases).
假设您的图像或帧的边界上有一个特征，而在下一个图像或帧中，该特征消失了。在下一帧中找不到该特征，因此 status 中该特征的相应索引将为 0。否则，status 在该索引处将为 1。
err 值在 status 为 0 的地方没有定义。否则，错误基本上是 "how well do the neighboring pixels around the feature correspond between the two images/frames"。这在 flags 参数下更清楚一点：
OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (see
minEigThresholddescription); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.
L1距离就是window内的绝对差之和。所以这只是在 images/frames 中的特征周围查找 window 并计算每个 window 中相应像素之间的绝对差，然后将它们相加。所以这就是当标志设置为 not 时会发生的情况。如果是，文档指导我们看minEigThreshold参数的说明：
minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than
minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.
因此，使用标志，您可以看到每个特征的最小特征值——如果您想找出要使用的阈值，这很有用。在 OpenCV 文档 (Bouguet00) 中引用的 LK 特征跟踪器的金字塔实现的 original paper 中，它在第 3 节（特征选择）中详细说明了这意味着什么：
So far, we have described the tracking procedure that takes care of following a point u on an image I to another location v on another image J. However, we have not described means to select the point u on I in the first place. This step is called feature selection. It is very intuitive to approach the problem of feature selection once the mathematical ground for tracking is led out. Indeed, the central step of tracking is the computation of the optical flow vector η^k (see pseudo-code of algorithm in section 2.4). At that step, the G matrix is required to be invertible, or in other words, the minimum eigenvalue of G must be large enough (larger than a threshold). This characterizes pixels that are "easy to track".