Fusion of Hyperspectral and Multispectral Images by Convolutional Sparse Representation

Sparse representation (SR)-based methods have achieved numerous successes in the fusion of hyperspectral and multispectral images (HSIs and MSIs). However, in many SR-based fusion methods, due to patch dividing, it is hard to make pixel values across boundaries of contiguous patches be exactly consistent, which limits the ability to preserve scene details. To remedy such deficiency, a fusion framework is proposed for HSIs and MSIs by using convolutional sparse representation (FCS). This novel fusion method consists of three stages: 1) the spectral dictionary is trained by the convolutional sparse dictionary learning algorithm to extract spectral information from HSIs; 2) hyperspectral and multispectral transferring matrices are estimated to map HSIs and MSIs onto the space of high-resolution hyperspectral images (HR-HSIs); and 3) we construct the convolutional sparse fusion model for HR-HSIs. Different from those traditional patch-based SR fusion methods, the FCS method focuses on the whole images instead of dividing patches, which can suppress the limitation of scene detail preservation caused by sparse coding on independent patches. Also, it belongs to a kind of online learning without lots of training samples. The Pavia dataset and the Paris dataset are used to evaluate the performance of our method. Experimental results indicate that the FCS method achieves much fusion performance compared with commonly used and state-of-the-art algorithms.