Approximating nonlinear fading-memory operators using neural networks models

A fading-memory system is defined as a system whose response map has unique asymptotic properties over some set of inputs. It is shown that any discrete-time fading-memory system can be uniformly approximated arbitrarily closely over a compact set of input sequences by uniformly approximating either its external or internal representation sufficiently closely. In other words, the problem of uniformly approximating a fading-memory system reduces to the problem of uniformly approximating continuous real-valued functions on compact sets. The perceptron is shown to realize a set of continuous real-valued functions that is uniformly dense on compacta in the set of all continuous functions. Using the perceptron to uniformly approximate the external and internal representations of a fading-memory system results, respectively, in simple nonlinear finite-memory and infinite-memory system models.