Constrained adversarial loss for generative adversarial network‐based faithful image restoration

Generative adversarial networks (GAN) have been successfully used in many image restoration tasks, including image denoising, super‐resolution, and compression artifact reduction. By fully exploiting its characteristics, state‐of‐the‐art image restoration techniques can be used to generate images with photorealistic details. However, there are many applications that require faithful rather than visually appealing image reconstruction, such as medical imaging, surveillance, and video coding. We found that previous GAN‐training methods that used a loss function in the form of a weighted sum of fidelity and adversarial loss fails to reduce fidelity loss. This results in non‐negligible degradation of the objective image quality, including peak signal‐to‐noise ratio. Our approach is to alternate between fidelity and adversarial loss in a way that the minimization of adversarial loss does not deteriorate the fidelity. Experimental results on compression‐artifact reduction and super‐resolution tasks show that the proposed method can perform faithful and photorealistic image restoration. Generative adversarial networks (GANs) have shown promising results for image restoration as well as image generation. However, previous GAN‐based methods based on a loss function in the form of a weighted sum of the fidelity loss and the adversarial loss tend to fail when balancing the two losses. The proposed method accepts the parameter update of a network for an adversarial loss if the update also reduces the fidelity loss. Experimental results on compression‐artifact reduction and super‐resolution tasks show that the proposed approach can perform reliable and photorealistic image restoration.