cv2 Farneback Optical FLow值太低

Question

我正在尝试计算两个帧之间的光通量，然后使用计算出的光流扭曲前一帧。我发现cv2具有Farneback Optical FLow，因此我正在使用它来计算Flow。我从cv2 tutorial中获取了默认参数，然后使用this answer中提供的代码对框架进行了变形。但是，当我看到扭曲的帧时，它与前一帧完全一样，并且没有变化（数组相等）。

通过进一步的调试，我发现计算出的流量值太低。为什么会这样呢？我做错什么了吗？

代码：

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = skimage.color.rgb2gray(prev_frame)
    next_gray = skimage.color.rgb2gray(next_frame)
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow


def warp_frame(prev_frame: numpy.ndarray, flow: numpy.ndarray):
    h, w = flow.shape[:2]
    flow = -flow
    flow[:,:,0] += numpy.arange(w)
    flow[:,:,1] += numpy.arange(h)[:,numpy.newaxis]
    # res = cv2.remap(img, flow, None, cv2.INTER_LINEAR)
    next_frame = cv2.remap(prev_frame, flow, None, cv2.INTER_LINEAR)
    return next_frame


def demo1():
    prev_frame_path = Path('./frame025.png')
    next_frame_path = Path('./frame027.png')
    prev_frame = skimage.io.imread(prev_frame_path.as_posix())
    next_frame = skimage.io.imread(next_frame_path.as_posix())
    flow = get_optical_flow(prev_frame, next_frame)
    print(f'Flow: max:{flow.max()}, min:{flow.min()}, mean:{flow.__abs__().mean()}')
    warped_frame = warp_frame(prev_frame, flow)

    print(numpy.array_equal(prev_frame, warped_frame))

    pyplot.subplot(1,3,1)
    pyplot.imshow(prev_frame)
    pyplot.subplot(1,3,2)
    pyplot.imshow(next_frame)
    pyplot.subplot(1,3,3)
    pyplot.imshow(warped_frame)
    pyplot.show()
    return

输入图像：

输出：扭曲图像与上一幅图像完全相同，而后一幅图像应看起来像下一幅图像。

感谢您的任何帮助！

Answer 1

问题在于将rgb帧转换为灰色。 skimage.color.rgb2gray()将强度范围从[0,255]更改为[0,1]。将其更改回[0,255]可行！

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = (skimage.color.rgb2gray(prev_frame) * 255).astype('uint8')
    next_gray = (skimage.color.rgb2gray(next_frame) * 255).astype('uint8')
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow