Robust frontal view search using extended manifold learning

•Propose an effective method to search the frontal view in face image sequence.•Present a pairwise K-nearest neighbor protocol to extend manifold learning.•Present localized edge orientation histogram for face image in manifold learning. Many 2D face processing algorithms can perform better using frontal or near frontal faces. In this paper, we present a robust frontal view search method based on manifold learning, with the assumption that with the pose being the only variable, face images should lie in a smooth and low-dimensional manifold. In 2D embedding, we find that manifold geometry of face images with varying poses has the shape of a parabola with the frontal view in the vertex. However, background clutter and illumination variations make frontal view deviate from the vertex. To address this problem, we propose a pairwise K-nearest neighbor protocol to extend manifold learning. In addition, we present an illumination-robust localized edge orientation histogram to represent face image in the extended manifold learning. The experimental results show that the extended algorithms have higher search accuracy, even under varying illuminations.