Multi-view 3D data fusion and patching to reduce Shannon entropy in Robotic Vision

Optical Sensors Fusion is intended to enrich the data obtained from Robotic Vision systems, which play a crucial role in applications such as machine guidance and monitoring. This paper presents a data augmentation method that uniquely combines cameras with the rotational wide-based Laser Scanner Technical Vision System (LSTVS), an innovative system not previously explored in conjunction with other 3D data acquisition systems. This combined work aims to address common issues in 3D reconstruction by utilizing a deterministic position estimation from the laser scanner complementary to probabilistic estimations from stereo vision. This approach aims to reduce informational entropy in regions where data is lacking or difficult to interpret, primarily due to the inherent limitations of Robotic Vision systems. An LSTVS with stereo cameras prototype was calibrated using intrinsic and extrinsic parameters of the cameras and laser scanner components, enabling laser positioning over selected interest point and areas from the stereo 3D data. By fusing 3D data from both systems, data quality is improved on challenging surfaces often problematic for stereo vision, like low texture or non-Lambertian surfaces. Experiments aim to test the stereo system limits in order to fuse the obtained data. Multiple experiments with variable parameters (angle of view, striking distance, most indicative kinds of the obstacle's refractive surface among them) are described in order to prove new abilities of the proposed combined RV. This improvement is made possible by using complementary data from LSTVS and original smart algorithm of image patching, as shown in the experimental results. •Is proposed a combination of stereo vision and LSTVS for reduction of Shannon entropy.•Methodology for collaborative working and information fusion considering their functional principles is presented.•Both systems are tested working together using a prototype and practical results are analyzed.•Combination of laser scanner and stereo system achieves an effective and accurate position estimation and information fusion.