Cost-effective real-time recognition for human emotion-age-gender using deep learning with normalized facial cropping preprocess

Because of technological advancement, human face recognition has been commonly applied in various fields. There are some HCI-related applications, such as camera-ready chatbot and companion robot, require gathering more information from user’s face. In this paper, we developed a system called EAGR for emotion, age, and gender recognition, which can perceive user’s emotion, age and gender based on the face detection. The EAGR system first applies normalized facial cropping (NFC) as a preprocessing method for training data before data augmentation, then uses convolution neural network (CNN) as three training models for recognizing seven emotions (six basics plus one neutral emotion), four age groups, and two genders. For better emotion recognition, the NFC will extract facial features without hair retained. On the other hand, the NFC will extract facial features with hair retained for better age and gender recognition. The experiments were conducted on these three training models of emotion, age and gender recognitions. The recognition performance results from the testing dataset, which has been normalized for tilted head by proposed binocular line angle correction (BLAC), showed that the optimal mean accuracy rates of real-time recognition for seven emotions, four age groups and two genders were 82.4%, 74.95%, and 96.65% respectively. Furthermore, the training time can be substantially reduced via NFC preprocessing. Therefore, we believe that EAGR system is cost-effective in recognizing human emotions, ages, and genders. The EAGR system can be further applied in social applications to help HCI service provide more accurate feedback from pluralistic facial classifications.