Robust spatiotemporal stereo against image motion and temporal disparity variation

When there exists camera motion as well as scene motion, the position and disparity of a pixel in an image sequence temporally vary with time. Such image motion (IM) and temporal disparity variation (TDV) degrade the performance of spatiotemporal stereo matching. In this Letter, a robust spatiotemporal similarity measure that addresses IM and TDV is proposed. To this end, an irregular spatiotemporal window whose temporal windows are located by a TDV value and optical flows is designed. In addition, a spatiotemporal guided filter-based aggregation technique using temporal weights based on temporal proximity and flow reliability is presented. To handle a large number of labels effectively, a search range reduction method for finding a probable label set is presented. Experimental results show that the proposed method yields consistent and accurate disparity maps under IM and TDV.