Correlation Based Online Dictionary Learning Algorithm

The goal of dictionary learning algorithms is to learn a set of atoms called dictionary from a set of training data such that each training data can be represented sparsely by the dictionary. Most dictionary learning algorithms use two alternating steps, sparse coding and dictionary update, to solve this problem. In this paper, we propose a new online dictionary learning algorithm with a novel dictionary update step. In the new update method, only the atoms involved in the sparse representation of new training data are adaptively updated. The adaptive update also includes using the previous training data that have common atoms with the new training data in their sparse representation. The proposed algorithm reduces the computational complexity by reducing the number of atoms updating at each iteration and the number of training data contributing in the dictionary update. In the proposed algorithm, different weights are given to training data, so each has the proper level of influence in the dictionary update. The simulation results show this improves the performance of the algorithm, in convergence speed and representation error. We also show that our algorithm has the ability of full recovery of dictionary with only one trial over training data and does not need any dictionary pruning to remove the unused atoms of the dictionary. We compare our proposed algorithm with well-known online and batch based methods using synthetic and autoregressive data.