Diversifying detail and appearance in sketch-based face image synthesis

Sketch-based face image synthesis has gained greater attention with the increasing realism of its output images. However, existing studies have overlooked the significance of output diversity : because sketches are inherently ambiguous, it would be desirable to have various output candidates for a single-input sketch. In this paper, we explore synthesis of diverse face images from a single sketch by using a three-stage framework consisting of sketch refinement, detail enhancement, and appearance synthesis. Each stage uses supervised learning with neural networks. With this three-stage framework, we can separately control the detail (e.g., wrinkles and hair structures) and appearance (e.g., skin and hair colors) of output face images separately by using multiple latent codes. Quantitative and quantitative evaluations demonstrate that our method offers greater diversity in its output images than the state-of-the-art methods, while retaining the output realism.