A Dual-Mode Intensity and Polarized Imaging System for Assisting Autonomous Driving

Multimodal image fusion is beneficial to improve the target contrast. Taking advantage of the significant polarization characteristics of vehicles, the fusion images combining polarization and intensity characteristics will help the autonomous driving system to find and identify target objects with low contrast. To this end, in this article, we propose a polarization-intensity image imaging system for autonomous driving, including hardware as well as algorithm part. The hardware part consists of a polarization camera, which is used to capture the degree of linear polarization (DoLP) and intensity images. The two kinds of images are fused by the proposed algorithm, named as hierarchical processing fusion network (HPFusion), to highlight the low contrast target. HPFusion is a hierarchical processing fusion network with multiscale encoder-decoder architecture, in which different modules are used to extract the features of different depths in the encoder subnetwork. Dense jump connections are used in decoder networks to take full advantage of the characteristics of different layers and scales. In the fusion module, the methods of addition and splicing are used to combine the features of different scales from space and channel dimensions and realize end-to-end image fusion at the same time. Our experiments on our own captured datasets show that our HPFusion can maintain high brightness and rich texture details of polarized objects for better scene representation and visual perception; in addition, our HPFusion achieves better fusion performance, surpassing other advanced methods in qualitative and quantitative comparison methods. The vehicle detection results show that our system has a certain application prospect in autonomous driving.