A Measure of Semantic Class Difference of Point Reprojection Pairs in Camera Pose Estimation

In this paper, we propose a new measure that evaluates the semantic errors of camera poses in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM). Traditionally, VO/VSLAM methods have used photometric images to estimate camera poses, but they suffer from varying illumination and viewpoint changes. Thus, methods using semantic images have been an alternative to increase consistency, as semantic information has shown its robustness even in hostile environments. Our measure compares semantic classes of map point reprojection pairs between images to improve the camera pose estimation accuracy in VO/VSLAM. To evaluate the difference between semantic classes, we adopt the normalized information distance (NID) from information theory. Furthermore, we suggest a weight parameter to balance the existing error of VO/VSLAM with the semantic error introduced by our approach. Our experimental results, obtained from the VKITTI and KITTI benchmark datasets, show that the proposed semantic error measure reduces both the relative pose error (RPE) and absolute trajectory error (ATE) of camera pose estimation compared to the existing photometric image-based errors of indirect and direct VO/VSLAM.