Unsupervised missing information reconstruction for single remote sensing image with Deep Code Regression

•We propose Deep Code Regression to reconstruct gaps in remote sensing images.•The method can directly operate on the target in an unsupervised manner.•The method can deal with various kinds of gaps in a unified framework. Remote sensing images have been applied to many aspects in Earth observation work. However, tons of optical remote sensing images are abandoned due to the information loss caused by the clouds and damage of sensing instruments. Recently, many deep learning methods have been proposed to reconstruct the missing information of remote sensing images but they will be non-effective when it comes to the condition where there is no training dataset. In this paper, we propose an unsupervised method which can reconstruct single remote sensing image without training datasets in a deep neural network. The main idea is to process a reference image of the corrupted image with a deep self-regression network and extract the internal map, which possesses the same spatial information as the reference image. The residual information of the corrupted image is used to constrain the spectral authority of internal map to obtain the reconstruction results. We apply the proposed method in three conditions: 1) dead pixel reconstruction, 2) multitemporal reconstruction and 3) heterogeneous data reconstruction. We conduct simulation experiments and real data experiments in three conditions to confirm the superiority of our methods. The results show that the proposed method outperforms some state-of-the-art algorithms.