cv2 Farneback 光流量值太低

Question

我正在尝试计算两帧之间的光流，然后使用计算出的光流扭曲前一帧。我发现 cv2 有 Farneback Optical FLow，所以我用它来计算 Flow。我从 cv2 tutorial and I'm warping the frame using the code given in this answer 中获取了默认参数。但是当我看到扭曲的帧时，它与前一帧完全一样并且没有变化（数组相等）。

进一步调试，发现计算出来的流量值太小了。为什么会这样？我做错了什么吗？

代码:

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = skimage.color.rgb2gray(prev_frame)
    next_gray = skimage.color.rgb2gray(next_frame)
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow


def warp_frame(prev_frame: numpy.ndarray, flow: numpy.ndarray):
    h, w = flow.shape[:2]
    flow = -flow
    flow[:,:,0] += numpy.arange(w)
    flow[:,:,1] += numpy.arange(h)[:,numpy.newaxis]
    # res = cv2.remap(img, flow, None, cv2.INTER_LINEAR)
    next_frame = cv2.remap(prev_frame, flow, None, cv2.INTER_LINEAR)
    return next_frame


def demo1():
    prev_frame_path = Path('./frame025.png')
    next_frame_path = Path('./frame027.png')
    prev_frame = skimage.io.imread(prev_frame_path.as_posix())
    next_frame = skimage.io.imread(next_frame_path.as_posix())
    flow = get_optical_flow(prev_frame, next_frame)
    print(f'Flow: max:{flow.max()}, min:{flow.min()}, mean:{flow.__abs__().mean()}')
    warped_frame = warp_frame(prev_frame, flow)

    print(numpy.array_equal(prev_frame, warped_frame))

    pyplot.subplot(1,3,1)
    pyplot.imshow(prev_frame)
    pyplot.subplot(1,3,2)
    pyplot.imshow(next_frame)
    pyplot.subplot(1,3,3)
    pyplot.imshow(warped_frame)
    pyplot.show()
    return

输入图像：

输出:
扭曲的图像与上一张图像完全相同，但它应该看起来像下一张图像。

感谢任何帮助！

Answer 1

问题在于将 rgb 帧转换为灰色。 skimage.color.rgb2gray() 将强度范围从 [0,255] 更改为 [0,1]。将其改回 [0,255] 成功了！

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = (skimage.color.rgb2gray(prev_frame) * 255).astype('uint8')
    next_gray = (skimage.color.rgb2gray(next_frame) * 255).astype('uint8')
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow

cv2 Farneback 光流量值太低

cv2 Farneback Optical FLow values are too low

python

opencv

opticalflow