Shallow multi-branch attention convolutional neural network for micro-expression recognition

Micro-expression recognition (MER) has become challenging because it is difficult to extract subtle facial variations of micro-expressions (MEs). Several approaches have been recently exploited to model MER’s global expression features directly. However, these methods do not identify discriminative ME representations, resulting in suboptimal performance. To address the problem of localization and asymmetry in ME movements, we propose a novel shallow multi-branch attention convolutional neural network (SMBANet) to recognize MEs. SMBANet seeks to obtain faintly distinctive ME characteristics and it contains three components: region division, local expression feature learning, and global feature fusion. First, region division partitions facial areas into four regions, i.e., upper left, upper right, lower left, and lower right half-face. Second, four branches with inception module and proposed efficient residual channel spatial (ERCS) attention module are designed to learn local expression features. Last, ME labels are predicted via global fusion with adaptively weighting four branches’ features. Experiments conducted on the composite database published by MEGC 2019 validate the effectiveness of SMBANet under the composite database evaluation protocol. The results show that SMBANet yields salient and discriminative ME representations and achieves more competitive performance than comparable state-of-the-art methods for MER. And ablation experiments’ results exhibit that our proposed ERCS attention outperforms the classical attention module, i.e., ECA and CBAM.