Towards generalized morphing attack detection by learning residuals

•Presents a new approach for Generalizable Morphing Attack Detection by learning residuals from Encoder-Decoder network.•Proposes to explore complementary color spaces for learning morphing cues in an end-to-end manner.•Extensive experiments on 5 different databases with 2 landmark-based & 3 Generative Adversarial Network (GAN) based morphing.•Detailed explainability analysis of proposed approach using three different Class Activation Maps (CAM) for MAD. Face recognition systems (FRS) are vulnerable to different kinds of attacks. Morphing attack combines multiple face images to obtain a single face image that can verify equally against all contributing subjects. Various Morphing Attack Detection (MAD) algorithms have been proposed in recent years albeit limited generalizability. We present a new approach for MAD in this work with better generalization than state-of-the-art (SOTA) algorithms. We propose an end-to-end multi-stage encoder-decoder network for learning the residuals of morphing process to detect attacks. Leveraging the residuals, we learn an efficient classifier using cross-entropy loss and asymmetric loss. The use of asymmetric loss in our approach is motivated by imbalanced distribution of morphs and bona fides. An extensive set of experiments are conducted on five different datasets consisting of two landmark based and three Generative Adversarial Network (GAN) based morphs in various settings such as digital, print-scan and print-scan-compression. We first demonstrate a near-ideal performance of the proposed MAD with Detection Equal Error Rate (D-EER) of 0% in the best case and 2.58% in the worst case in the digital domain in closed-set protocol, i.e., known attacks. Further, we demonstrate the applicability of the proposed approach on 60 different combinations where the testing set contains unknown morphing attacks in open-set protocol to illustrate the generalization ability of our proposed approach. Through training the proposed approach on landmark-based morph generation data alone, we obtain an EER of 3.59% in the best case and 12.89% in the worst case for morphed images in the digital domain, reducing the error rates from 45.67% and 30.23% respectively, in open-set protocol. We further present an extensive analysis of the proposed approach through Class Activation Maps (CAM) to explain the decisions using by making use of three complementary CAM analysis.