A saliency-based multiscale approach for infrared and visible image fusion

•A multi-scale decomposition framework is proposed to address blurred edge in image fusion.•The weight maps for fusion of salient layers are calculated by a nonlinear function.•Phase congruency is adopted to transmit more detail into the fused detail layer.•A total variation minimization model is proposed to eliminate the influence by low light. The ideal fusion of the infrared image and visual image should integrate complete bright features of the infrared image, and preserve original visual information of the visual image as much as possible. To this end, we propose a multi-scale decomposition fusion method based on saliency. In particular, the saliency detection and a Gaussian smoothing filter are first employed to decompose source images into salient layers, detail layers and base layers. Then we adopt a nonlinear function to calculate the weight coefficient to fuse salient layers and highlight the target. Subsequently, we use a fusion rule based on phase congruency for fusion of detail layers so that the details could be retained better than the traditional “max-absolute” fusion rule. Experiments show that the proposed method can achieve better fusion effect than the state-of-the-art methods qualitatively and quantitatively. Moreover, for the ill-illumination fused image, in order to get better visual effect, we further propose a contrast enhancement algorithm based on total variation minimization. Experiments show that the proposed method can enhance the contrast and retain details of the source images well.