RFRFlow: Recurrent Feature Refinement Network for Optical Flow Estimation

Optical flow represents vector motion of targets, and its estimation is a hot research in computer vision. The traditional prior-based methods build mathematical models using different regularizers on object characteristics. These methods are straightforward and easily understood, but they suffer from a high-computational burden. Deep learning-based approaches greatly improve their efficiency, but their filter sizes are fixed and receptive fields (RFs) are large. To solve these problems, this article proposes a novel recurrent feature refinement network based on short-term dense connection modules for optical flow estimation (RFRFlow), including the following key parts: first, we utilize short-term dense concatenate (STDC) modules as a backbone part for constructing a feature extracting network, which can preserve rich details and use global averaging pooling to extract global context information with a large RF. Second, a 4-D correlation volume is constructed using context features of image pairs to extract multiscale 4-D correlation information. Third, a motion feature encoder (MFE) is reported to transfer the multiscale 4-D correlation information into 2-D motion features. Finally, 2-D context features and 2-D motion features are input into a global motion aggregation (GMA) decoder for pursuing the final optical flow. In the experiments, we first present and analyze the results of ablation studies on the selection of batch size and image size. Then, both quantitative and qualitative results on Sintel and KITTI datasets are compared, showing that the proposed RFRFlow is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and objects' shape preservation.