A complementary regression network for accurate face alignment

This paper proposes a complementary regression network (CRN) that combines global and local regression methods to align faces. A global regression network (GRN) generates the coordinates of facial landmark points directly such that all facial feature points are fitted to the input face on the whole and a local regression network (LRN) generates the heatmap of facial landmark points such that each channel localizes the detail of its facial landmark point well. The CRN converts the GRN's coordinates to another heatmap, then uses with the LRN's heatmap to get the final facial landmark points. The CRN works complementarily such that the GRN's overall fitting tendency compensates for the LRN's poor alignment caused by missing local information, whereas the LRN's detailed representation compensates for the GRN's poor alignment caused by global miss-fitting. We conducted several experiments on the 300-W public dataset, the 300-W private dataset, and the Menpo dataset and the proposed CRN achieved 3.14%, 3.74%, and 1.996% the-state-of-art face alignment accuracy in terms of percentage of normalized mean error, respectively. •We propose CRN that combines global and local regression networks for face alignment.•CRN converts GRN's coordinates to LRN's heatmap, vice versa, to get landmark points.•CRN works complementarily that GRN and LRN compensate their demerits each other.•We conducted several experiments on 300-W public/private dataset Menpo dataset.•We achieved 3.14%, 3.74%, 2% SOTA face alignment accuracy in terms of PNME.