A Self-Attention Based Wasserstein Generative Adversarial Networks for Single Image Inpainting

With the popularization of portable devices such as mobile phones and cameras, digital images have been widely disseminated in human life. However, due to factors such as photoaging, shooting environment, etc., images will encounter some defects. To restore these defective images quickly and realistically, image inpainting technology emerges as the times require, and digital image processing technology has been rapidly developed. In recent years, thanks to the rapid development of deep learning, which has been gradually applied to image inpainting and has shown excellent performance. For the image inpainting problem, because the damaged images usually have a large number of missing areas, the inpainting models are required to have a stronger ability to mine the global correlation of the image itself and maintain the global consistency of the restoration patches. Given the above shortcomings, a self-attention-based Wasserstein generative adversarial networks (WGAN) single image inpainting method is proposed. First, WGAN is introduced, in which, the global consistency of the inpainting region is ensured through the learning of a generative adversarial model. Second, the proposed model uses the Wasserstein distance (Earth mover distance) to measure the similarity of the two distributions. Compared with the Kullback–Leibler (KL) divergence and the Jensen–Shannon (JS) divergence, even if the support sets of the two distributions do not overlap or overlap very little, the Wasserstein distance can still reflect the distance between the two distributions, and it is conducive to ensuring the stability of GAN. Third, self-attention is introduced to exploit the self-similarity of local features of images fully. The experimental results show that the proposed method can mine the global correlation of the image itself better than the compared methods in quantitative as well as qualitative assessments.