SELP: A general-purpose framework for learning the norms from saliencies in spatiotemporal data

Sensors that monitor around the clock are everywhere. Due to the sheer amount of data these sensors can generate, the resources required to store, protect personal information, and analyze them are enormous. Since noteworthy events happen only occasionally, it is not necessary to store or analyze the data generated at every instant of time. Rather, it is imperative for a smart memory to learn the norms in such data so that only the abnormal (or salient) events may be stored. We present a general-purpose biologically plausible computational framework, called SELP, for learning the norms (or invariances) as a hierarchy of features from space- and time-varying data in an unsupervised and online manner from saliencies or surprises in the data. Given streaming data, this framework runs a relentless cycle – detect unexpected or Salient event, Explain the salient event, Learn from its explanation, Predict the future events – involving the real external world and its internal model, and hence the name. Experimental results from different functions of this framework are presented with a particular emphasis on the role of lateral connections in each layer.