A batch/recursive algorithm for 3D scene reconstruction

We present a new formulation of sequential least-squares applied to scene and motion reconstruction from image features. We argue that recursive techniques will become more important both for real-time control applications and also interactive vision applications. The aim is to approximate as well as possible the result of the batch bundle adjustment method. In previously published work we described an algorithm which works well if the same features are visible throughout the sequence. Here we show how to deal with new features in a way that avoids deterioration of the results. The main theoretical advance here is showing how to adjust the system information matrix when scene/camera parameters are removed from the reconstruction. We show how this procedure affects the sparseness of the information matrix, and thus how to achieve an efficient recursive solution to the reconstruction problem.