Semi-supervised class-conditional image synthesis with Semantics-guided Adaptive Feature Transforms

Generative Adversarial Networks (GANs) have become the mainstream models for class-conditional synthesis of high-fidelity images. To reduce the demand for labeled data, we propose a class-conditional GAN with Semantic-guided Adaptive Feature Transforms, which is referred to as SAFT-GAN for semi-supervised image synthesis. Instead of simply incorporating a classifier to infer the class labels of unlabeled data, the key idea behind SAFT-GAN is to incorporate class-semantic guidance in real-fake discrimination. More specifically, we adopt a two-head architecture for a discriminator: A label-embedded head identifies real and fake instances, conditioned on class label. To focus more on class-related regions, we exploit class-aware attention information to regularize this head via regional feature transforms. On the other hand, to make better use of unlabeled data, we design a label-free head, on which channel-adaptive feature transforms are imposed to fuse the discriminator and classifier features, such that the class semantics of synthesized images can be improved. Extensive experiments are performed to demonstrate how class-conditional image synthesis can benefit from the proposed feature transforms, and also demonstrate the superiority of SAFT-GAN. •We design a semantic-guided discriminator for class-conditional image generation.•We find that classier features encapsulate instance-level semantic information.•Class-aware attention maps are useful for real-fake identification.•Jointly training with a classifier improves the class separability of synthesized data.