Asynchronous adaptive conditioning for visual–inertial SLAM

This paper is concerned with real-time monocular visual–inertial simultaneous localization and mapping (SLAM). In particular a tightly coupled nonlinear-optimization-based solution that can match the global optimal result in real time is proposed. The methodology is motivated by the requirement to produce a scale-correct visual map, in an optimization framework that is able to incorporate relocalization and loop closure constraints. Special attention is paid to achieve robustness to many real world difficulties, including degenerate motions and unobservablity. A variety of helpful techniques are used, including: a relative manifold representation, a minimal-state inverse depth parameterization, and robust non-metric initialization and tracking. Importantly, to enable real-time operation and robustness, a novel numerical dog-leg solver is presented that employs multi-threaded, asynchronous, adaptive conditioning. In this approach, the conditioning edges of the SLAM graph are adaptively identified and solved for both synchronously and asynchronously. In this way one thread focuses on a small number of temporally immediate parameters and hence constitute a natural “front-end”; the other thread adaptively focuses on larger portions of the SLAM problem, and hence is able to re-estimate past parameters in the presence of new information: an ability that is useful for self-calibration, during degenerate motions, or when bias and the direction of gravity are poorly observed. Experiments with real and simulated data for both indoor and outdoor scenarios demonstrate that asynchronous adaptive conditioning is accurate, and able to closely track the batch SLAM maximum likelihood solution in real time.