Learn Probability Distributions with The Contrastive Hebbian Algorithm. The Artificial Intelligence and Psychology Project

This paper presents a method for training connectional networks that adhere to the principles of graded, random, adaptive, and interactive propagation of information (GRAIN). While our analysis has bee motivated by our desire to find a learning algorithm that would work in this environment, we have succeeded in implementing a model that encompasses a large class of previous connectionist algorithms under the same theoretical principles and that expands the scope of problems they can learn. Stimulations show examples where GRAIN networks successfully approximate both discrete and continuous probability distributions, demonstrating that their scope extends beyond what can be learned by backpropagation networks or standard Boltzmann machines.