cv2.calcOpticalFlowPyrLK 添加新点

Question

问题

我正在尝试为 Lucas Kanade 光流跟踪方法中的跟踪添加自定义点。但是在将我的自定义点插入到自动计算的点数组后调用此函数时，我收到断言失败错误。

OpenCV Error: Assertion failed ((npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0) in calcOpticalFlowPyrLK, file /opt/opencv-3.1.0/opencv/modules/video/src/lkpyramid.cpp, line 1114 
Traceback (most recent call last):
  File "/opt/pycharm-community-2016.2.3/helpers/pydev/pydevd.py", line 1580, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/pycharm-community-2016.2.3/helpers/pydev/pydevd.py", line 964, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/pycharm-community-2016.2.3/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/work/12_Kyloren/Repository/kyloren/firmware/Python/modules/kanade.py", line 45, in <module>
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
cv2.error: /opt/opencv-3.1.0/opencv/modules/video/src/lkpyramid.cpp:1114: error: (-215) (npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function calcOpticalFlowPyrLK

代码

我首先像这样点击鼠标采样点：-

def sample_track_points(event, x,y, flags, param):
    global user_points
    if event == cv2.EVENT_LBUTTONDBLCLK:
        user_points = np.empty([1, 1, 2], dtype=float)
        user_points[0][0] = [x,y]


cv2.namedWindow("frame")
cv2.setMouseCallback("frame",sample_track_points)

然后我将它连接到从 goodFeaturesToTrack

自动创建的主要点数组

p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, **feature_params)

这就是我连接的方式

if len(user_points) > 0:
    p0 = np.concatenate([p0, user_points])
    user_points = np.empty([0,1,2])

在这一行失败

    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

我已经逐步调试并修复了所有 numpy 数组不匹配问题。所以问题出在一些内部机制上。

有人可以帮忙吗？

完整代码

我已经修改了这个程序here（第一个）下面是我完全修改的代码。

import numpy as np
import cv2
cap = cv2.VideoCapture("/work/12_Kyloren/Videos/WC5_Door_Front.mp4")
# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 100,
                       qualityLevel = 0.3,
                       minDistance = 7,
                       blockSize = 7 )
# Parameters for lucas kanade optical flow
lk_params = dict( winSize  = (15,15),
                  maxLevel = 2,
                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))


# Create some random colors
color = np.random.randint(0,255,(100,3))
# Take first frame and find corners in it
ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, **feature_params)

# Create a mask image for drawing purposes
mask = np.zeros_like(old_frame)


# add points to the existing array on mouseclick
user_points = np.empty([0,1,2],dtype=float)
def sample_track_points(event, x,y, flags, param):
    global user_points
    if event == cv2.EVENT_LBUTTONDBLCLK:
        user_points = np.empty([1, 1, 2], dtype=float)
        user_points[0][0] = [x,y]


# set the mouse call back
cv2.namedWindow("frame")
cv2.setMouseCallback("frame",sample_track_points)

# start the processing
while(1):
    ret,frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    if len(user_points) > 0:
        p0 = np.concatenate([p0, user_points])
        user_points = np.empty([0,1,2])

    # calculate optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
    # Select good points
    good_new = p1[st==1]
    good_old = p0[st==1]
    # draw the tracks
    for i,(new,old) in enumerate(zip(good_new,good_old)):
        a,b = new.ravel()
        c,d = old.ravel()
        mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)
        frame = cv2.circle(frame,(a,b),5,color[i].tolist(),-1)
    img = cv2.add(frame,mask)
    cv2.imshow('frame',img)
    k = cv2.waitKey(30) & 0xff

    if k == 27:
        break
    # Now update the previous frame and previous points
    old_gray = frame_gray.copy()
    p0 = good_new.reshape(-1,1,2)
cv2.destroyAllWindows()
cap.release()

重现错误

双击预览 window 将导致它因断言失败错误而崩溃。

Answer 1

问题出在数组的数据类型上。我提供的是 float 而不是 np.float32，这就是问题所在！它假定数组为 float64，而 openCV 拒绝了。

现在正确的代码变成了

# add points to the existing array on mouseclick
user_points = np.empty([0,1,2],dtype=np.float32)
def sample_track_points(event, x,y, flags, param):
    global user_points
    if event == cv2.EVENT_LBUTTONDBLCLK:
        user_points = np.empty([1, 1, 2], dtype=np.float32)
        user_points[0][0] = [x,y]


# set the mouse call back
cv2.namedWindow("frame")
cv2.setMouseCallback("frame",sample_track_points)

cv2.calcOpticalFlowPyrLK 添加新点

cv2.calcOpticalFlowPyrLK Adding New Points

python

opencv

numpy

opticalflow

cv2