Neural ARX Models and PAC Learning

The PAC learning theory creates a framework to assess the learning properties of models such as the required size of the training samples and the similarity between the training and training performances. These properties, along with stochastic stability, form the main characteristics of a typical dynamic ARX modeling using neural networks. In this paper, an extension of PAC learning theory is defined which includes ARX modeling tasks, and then based on the new learning theory the learning properties of a family of neural ARX models are evaluated. The issue of stochastic stability of such networks is also addressed. Finally, using the obtained results, a cost function is proposed that considers the learning properties as well as the stochastic stability of a sigmoid neural network and creates a balance between the testing and training performances.