DCCMF-GAN: double cycle consistently constrained multi-feature discrimination GAN for makeup transfer

Makeup transfer aims to transfer the makeup of the human face in the reference image to another face in the source image. Most mainstream makeup transfer methods directly learn the makeup, which often regard the interference information (posture, illumination, shadow and background) as part of makeup. We propose a novel Double Cycle Consistently constrained Multi-Feature Discrimination Generative Adversarial Networks (DCCMF-GAN) for makeup transfer, which is based on the separation of "makeup" and "content". DCCMF-GAN is nested by cycle reconstruction networks CycleI and CycleII, which are separately constrained by cycle consistency loss “cycle consistency1” and “cycle consistency2”. Both cycle networks consist of two makeup-transfer generators G and two multi-feature discriminators, respectively. G first encodes to separate "makeup" and "content" of the source and the reference images. Then fuses the source content with reference makeup for makeup application, and merges the source makeup with reference content for makeup removal. Under the constraint of double cycle consistency loss and the adversarial learning of multi-feature discriminators in terms of identity, global makeup and focused local makeup ensure to generate pleasant makeup conversion results. Compared with several state-of-the-art makeup transfer methods, the proposed method is insensitive to pose, illumination, shadow, expression, aging and other interference, which achieves high-quality makeup transfer with strong robustness.