A scalable optimization approach for fitting canonical tensor decompositions

Tensor decompositions are higher‐order analogues of matrix decompositions and have proven to be powerful tools for data analysis. In particular, we are interested in the canonical tensor decomposition, otherwise known as CANDECOMP/PARAFAC (CP), which expresses a tensor as the sum of component rank‐one tensors and is used in a multitude of applications such as chemometrics, signal processing, neuroscience and web analysis. The task of computing CP, however, can be difficult. The typical approach is based on alternating least‐squares (ALS) optimization, but it is not accurate in the case of overfactoring. High accuracy can be obtained by using nonlinear least‐squares (NLS) methods; the disadvantage is that NLS methods are much slower than ALS. In this paper, we propose the use of gradient‐based optimization methods. We discuss the mathematical calculation of the derivatives and show that they can be computed efficiently, at the same cost as one iteration of ALS. Computational experiments demonstrate that the gradient‐based optimization methods are more accurate than ALS and faster than NLS in terms of total computation time. Copyright © 2011 John Wiley & Sons, Ltd. Tensor decompositions are higher‐order analogues of matrix decompositions and have proven to be powerful tools for data analysis. For fitting the CANDECOMP/PARAFAC tensor decomposition, we propose the use of gradient‐based optimization methods such as nonlinear conjugate gradients. Computational experiments demonstrate that the gradient‐based optimization methods are more accurate than the standard alternating least‐squares (ALS) and faster than second‐order optimizaion in terms of total computation time.