Pseudo-LiDAR for Visual Odometry

As one of the important tasks in the field of robotics and machine vision, visual odometry provides tremendous help for various applications such as navigation, location, and so on. Conventionally, the task of visual odometry mainly relies on the input of continuous images. However, it is very complicated for the odometry network to learn the epipolar geometry information provided by the images. Since the 6-degree-of-freedom (DoF) pose transformation occurs in 3-D space and learning poses from 3-D point clouds are more straightforward, this article introduces the concept of pseudo-LiDAR to the odometry task. The pseudo-LiDAR point cloud is formed by backprojecting the depth map generated from the image into 3-D space. Due to the limitation of calculation power, most current algorithms based on the point cloud need to sample 8192 points from the point cloud as input, but such an approach makes the rich point cloud information in the pseudo-LiDAR point cloud not fully utilized. To address this problem, a projection-aware algorithm is adopted, which achieves efficient point cloud learning and improves the accuracy of the network while preserving the 3-D structure information in the pseudo-LiDAR point cloud. Finally, an image-only 2-D-3-D fusion module is proposed to enhance the pseudo-LiDAR point features using information such as the texture and color of the images. Through multimodal fusion, the network achieves a deeper understanding of the environment. Experiments on the KITTI dataset prove the effectiveness of our method. The source code will be open-sourced at https://github.com/IRMVLab/Pseudo-LiDAR-for-Visual-Odometry .