OpenCV 的密集光流 (Farneback) 函数的输出是什么？这如何用于在 Python 中构建光流图？

Question

我正在尝试使用 Opencv 的密集光流函数的输出来绘制运动矢量的箭袋图，但无法找到该函数实际输出的内容。这是代码：

import cv2
import numpy as np

cap = cv2.VideoCapture('GOPR1745.avi')

ret, frame1 = cap.read()
prvs = cv2.cvtColor(frame1,cv2.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)

hsv[...,1] = 255
count=0

while(1):
    ret, frame2 = cap.read()
    next = cv2.cvtColor(frame2,cv2.COLOR_BGR2GRAY)
    flow = cv2.calcOpticalFlowFarneback(prvs,next,None, 0.5, 3, 15, 3, 10, 1.2, 0)
    mag, ang = cv2.cartToPolar(flow[...,0], flow[...,1])

    hsv[...,0] = ang*180/np.pi/2
    hsv[...,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
    rgb = cv2.cvtColor(hsv,cv2.COLOR_HSV2BGR)
    if count==10:
        count=0

        print "flow",flow

    cv2.imshow('frame2',rgb)
    count=count+1
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
    elif k == ord('s'):
    prvs = next

cap.release()
cv2.destroyAllWindows()

这实际上与 OpenCv 密集光流教程中给出的代码相同。我从打印函数收到以下输出：

flow [[[  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  ..., 
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]]

 ..., 
 [[ -3.54891084e-14  -1.38642463e-14]
  [ -2.58058853e-14  -1.54020863e-14]
  [ -5.56561768e-14  -1.88019359e-14]
  ..., 
  [ -7.59403916e-15   1.16633225e-13]
  [  7.22156371e-14  -1.61951507e-13]
  [ -4.30715618e-15  -4.39530987e-14]]

 [[ -3.54891084e-14  -1.38642463e-14]
  [ -2.58058853e-14  -1.54020863e-14]
  [ -5.56561768e-14  -1.88019359e-14]
  ..., 
  [ -7.59403916e-15   1.16633225e-13]
  [  7.22156371e-14  -1.61951507e-13]
  [ -4.30715618e-15  -4.39530987e-14]]

我想知道这些值到底是什么？原始 X,Y 坐标？最后的 X,Y 坐标？移动距离？

我计划使用以下页面中的代码尝试找到初始坐标和最终坐标来制作箭袋图： https://www.getdatajoy.com/examples/python-plots/vector-fields 这是因为在 python 中没有我知道的函数可以为您绘制光流图。

提前致谢！

Answer 1

你快到了。让我们先看看那里说的calcOpticalFlowFarneback Documentation：

flow – computed flow image that has the same size as prev and type CV_32FC2.

所以你实际得到的是一个与输入帧大小相同的矩阵。
flow 矩阵中的每个元素都是一个点，表示该像素相对于 prev 帧的位移。这意味着您得到一个具有 x 和 y 值（以像素为单位）的点，它为您提供与最后一帧的 delta x 和 delta y。

Answer 2

我要劫持这个，因为它是同一个主题。

如果如@shravya 所述，单位是像素，为什么此代码不显示最大流量等于一？

我真的不懂单位

代码


import numpy as np
import cv2
import seaborn as sns

# Generating img
img = np.zeros(shape=(3,50,50)) # 3 time frames, 50x50

center0 = np.array((10,10))
center1 = np.array((30,30))
for each_time, each_x, each_y in itertools.product(range(img.shape[0]), range(img.shape[1]), range(img.shape[2])): 
    img[each_time, each_x, each_y] =  img[each_time, each_x, each_y] + 1000 *  1/(  0.1* ((center0[0]+each_time*displacement_x - each_x)**2 + 1*(center0[1]+each_time*displacement_y - each_y)**2)**0.5 + 1)    
    img[each_time, each_x, each_y] =  img[each_time, each_x, each_y] + 1000 * 1/(  0.1* ((center1[0]+each_time*displacement_x - each_x)**2 + 1*(center1[1]+each_time*displacement_y - each_y)**2)**0.5 + 1)


img = (img - img.min())/(img.max()-img.min()) # Normalizing

## Ploting
fig, axs = plt.subplots(ncols=3, squeeze=True, figsize=(20,5))
for i in range(3):
    im = sns.heatmap(img[i,:,:], ax = axs[i], vmin=0, vmax=np.max(img))

fig.suptitle('Image') 

def calc_flow(img):

    ## Optical flow
    img = img.astype(np.int16)

    prev = np.zeros(img[0, :, :].shape).astype(np.int16)
    flows = np.zeros(shape=(img.shape[0], img.shape[1], img.shape[2], 2))

    for i, each_frame in enumerate(img):
        if i > img.shape[0]:
            break

        next_ = each_frame
        flow = cv2.calcOpticalFlowFarneback(prev, next_, None,
                                           pyr_scale = 0.5,
                                            levels = 3,
                                            winsize = 12,
                                            iterations = 5,
                                            poly_n = 5,
                                            poly_sigma = 1.2,
                                            flags = 0) 

        flows[i, :, :, 0] = flow[..., 0]
        flows[i, :, :, 1] = flow[..., 1]

        prev = next_
    
    return flows

flow = calc_flow(img)

fig, axs = plt.subplots(ncols=3, nrows=2, squeeze=True, figsize=(20,10))
for i in range(3):
    im = sns.heatmap(flow[i,:,:, 0] ,ax = axs[0,i], vmin=0, vmax = np.max(flow)) 
    im = sns.heatmap(flow[i,:,:, 1] ,ax = axs[1,i], vmin=0, vmax = np.max(flow)) 
    
fig.suptitle('Flow x and y plots')

mag_img, pha_img = cv2.cartToPolar(flow[..., 0], flow[..., 1]) 

fig, axs = plt.subplots(ncols=3, nrows=2, squeeze=True, figsize=(20,10))
for i in range(3):
    im = sns.heatmap(mag_img[i,:,:], ax=axs[0,i], vmin=0, vmax = np.max(mag_img)) 
    im = sns.heatmap(pha_img[i,:,:], ax=axs[1,i], vmin=0, vmax = np.max(pha_img))  
    
fig.suptitle('Magnitude and phase plots')

## Output
print(flow.max()) # This should be equal to displacement!
print(np.abs(flow).min()) # this should be zero


print(mag_img.max()) # This should be equal to displacement!
print(mag_img.min()) # this should be zero

OpenCV 的密集光流 (Farneback) 函数的输出是什么？这如何用于在 Python 中构建光流图？

What is output from OpenCV's Dense optical flow (Farneback) function? How can this be used to build an optical flow map in Python?

python

python-2.7

opencv

motion-detection

opticalflow

代码