A total variation global optimization framework and its application on infrared and visible image fusion

The main bottleneck faced by total variation methods for image fusion is that it is difficult to design a novel optimization model that can be solved by numerical methods. This paper proposes a general framework of total variation optimized by deep learning for infrared and visible image fusion, which combines the advantages of deep convolutional neural networks. Under this framework, any arbitrary convex or non-convex total variation model for image fusion can be designed, and its optimization solution can be obtained through neural network learning. The core idea of the proposed framework is to transform the designed variational model into a loss function of a deep convolutional neural network, and then use the initial fused image of a source image and the output fused image to represent the data item, use the output image and the source image to represent the regularization term, and finally use a deep neural network learning method to obtain the optimal fused image. Based on the proposed framework, further research on pre-fusion, network model and regularization item can be carried out. To verify the effectiveness of the proposed framework, we designed a specific non-convex total variational model and performed experiments on the infrared and visible image datasets. Experimental results show that the proposed method has strong robustness, and compared with the fused images obtained by current state-of-art algorithms in terms of objective evaluation metrics and visual effects, the fused image obtained by the proposed method has more competitive advantages. Our code is publicly available at https://github.com/gzsds/globaloptimizationimagefusion .