Regularized least squares locality preserving projections with applications to image recognition

Locality preserving projection (LPP), as a well-known technique for dimensionality reduction, is designed to preserve the local structure of the original samples which usually lie on a low-dimensional manifold in the real world. However, it suffers from the undersampled or small-sample-size problem, when the dimension of the features is larger than the number of samples which causes the corresponding generalized eigenvalue problem to be ill-posed. To address this problem, we show that LPP is equivalent to a multivariate linear regression under a mild condition, and establish the connection between LPP and a least squares problem with multiple columns on the right-hand side. Based on the developed connection, we propose two regularized least squares methods for solving LPP. Experimental results on real-world databases illustrate the performance of our methods. •It is shown that LPP is equivalent to a multivariate linear regression under a mild condition.•A connection between LPP and a least squares problem is established.•Based on the developed connection, two regularized least squares methods for solving LPP are proposed.•The relationships among these two regularization methods and Laplacianface are analyzed.•Experimental results illustrate the outperformances of the proposed regularization methods.