Linear two-view multiple plane geometry

In this contribution we present a method for direct linear multiple plane estimation and optimization. The application is the detection of independent motion by background subtraction or temporal differences using image stabilization. Given a set of consistent homographies and a corresponding image segmentation for an image pair, it is possible to improve the image stabilization by locally warping. We show that linear constraints can be induced to homography estimation to ensure consistency. In this context consistency means that the homographies correspond to the same relative orientation but to different 3D planes. The relative orientation can be derived by measurements of additional sensors, e.g., inertial navigation system, odometry, or can be computed from the fundamental or essential matrix. The method is compared to standard estimation of homographies with simulated data, and the capability of the method is shown using depth estimation on the Middlebury-Stereo Benchmark dataset.