Optical Flow Problem in the Presence of Spatially-Varying Motion Blur

M. Hossein Daraei and Peyman Milanfar


Abstract

The problem of optical flow computation has various applications in Computer Vision, and serves as a key problem that has been well studied over the past decades. While most of the techniques for inferring optical flow are based on the brightness constancy assumption, various conditions including the presence of motion blur evidently violate this fundamental presumption. In low illumination scenarios and other conditions under which the shutter must be kept open for a relatively long interval, motion blur artifacts are inevitable. If the source image and the target image appear to be dissimilar due to different blur kernels, traditional methods will fail to achieve accurate results. After exploring advantages and shortcomings of various optical flow methods, e.g. CLG, Black-Anandan, and BlurFlow, we address the problem of optical flow in the presence of motion blur. In particular, we present a new approach that considers constructing a new pair of blurred frames, followed by regular optical flow computation. The proposed method, MB-CLG, eliminates the effect of non-uniform blur levels over the sequence. A proof is also provided to show the estimated flows are roughly equal to the ground truth flows that match the latent frames. The key observation is that if we applied the blur functions of the source image to the target image and vice versa, the brightness constancy assumption would be valid for the new frames. The proposed method employs a coarse-to-fine approach, in conjunction with a smoothness matrix to account for moving objects and occluded regions. Rather than warping frames or precomputing a large grid of derivatives as in Portz et al, MB-CLG directly warps the flows in the optimization process. This leads to lower computational cost, and requires less data storage. Based on the results for various synthetic sequences, MB-CLG outperforms existing optical flow algorithms in the sense of AAE, AEP and MSE.

Experimental Results

In the first example, we consider a synthetic sequence generated by warping the benchmark cameraman image, and applying motion blur deteriorations. Flow fields are constructed according to predefined homographies. Different methods are then applied on the video sequence to infer the underlying flow fields. Traditional methods, e.g. CLG, that do not account for motion blur artifacts result in deformed objects. BlurFlow also results in some other deformations: the white line is thinner compared to the ground truth. The proposed method, MB-CLG, achieves better results as evident in the magnified patches.


The second example is another artificial sequence that includes a moving object. We compare the results of MB-CLG with those of CLG, B&A, and BlurFlow. MB-CLG also employs a method to handle occluded regions and moving objects. Aside from subjective comparisons, we also measure the similarity of estimated flows and ground truth in terms of Average Endpoint Error (AEP) and Average Angular Error (AAE). Below are 6 sample frames in the "birds" sequence, and the corresponding visualization for estimated flows.

Paper

  download paper

@inproceedings{daraei2014optical,
  title={Optical Flow Computation in the Presence of Spatially-Varying Motion Blur},
  author={Daraei, Mohammad Hossein},
  booktitle={International Symposium on Visual Computing},
  pages={140--150},
  year={2014},
  organization={Springer}}