Independent component analysis using Potts models

We explore the extending application of Potts encoding to the task of independent component analysis, which primarily deals with the problem of minimizing the Kullback-Leibler divergence between the joint distribution and the product of all marginal distributions of output components. The competitive mechanism of Potts neurons is used to encode the overlapping projections from observations to output components. Based on these projections, the marginal distributions and the entropy of output components are made tractable for computation and the adaptation of the de-mixing matrix toward independent output components is obtained. The Potts model for ICA is well formulated by an objective function subject to a set of constraints, which leads to a novel energy function. A hybrid of the mean field annealing and the gradient descent method is applied to the energy function. Our approach to independent component analysis presents a new criterion for ICA. The performance of the Potts model for ICA given by our numerical simulations is encouraging.