FPGAN: Face de-identification method with generative adversarial networks for social robots

In this paper, we propose a new face de-identification method based on generative adversarial network (GAN) to protect visual facial privacy, which is an end-to-end method (herein, FPGAN). First, we propose FPGAN and mathematically prove its convergence. Then, a generator with an improved U-Net is used to enhance the quality of the generated image, and two discriminators with a seven-layer network architecture are designed to strengthen the feature extraction ability of FPGAN. Subsequently, we propose the pixel loss, content loss, adversarial loss functions and optimization strategy to guarantee the performance of FPGAN. In our experiments, we applied FPGAN to face de-identification in social robots and analyzed the related conditions that could affect the model. Moreover, we proposed a new face de-identification evaluation protocol to check the performance of the model. This protocol can be used for the evaluation of face de-identification and privacy protection. Finally, we tested our model and four other methods on the CelebA, MORPH, RaFD, and FBDe datasets. The results of the experiments show that FPGAN outperforms the baseline methods. •We proposed an end-to-end method for face de-identification, with one generator and dual discriminators.•We designed the pixel loss and content loss functions to retain partial links between the de-identified and the original images.•We improved the U-Net and used it as a generator to generate a sufficiently realistic face image.•We proposed new discriminators to improve the discrimination accuracy.•We applied the FPGAN to face de-identification of social robots and proposed the privacy protection system.