Learning Algorithms for Neural Networks and Neuro-Fuzzy Systems with Separable Structures

This article considers the problem of training neural networks and neuro-fuzzy systems, which lead to separable models, i.e., structures nonlinear with respect to some unknown parameters and linear with respect to others. New algorithms for training them are proposed that are based on a nonlinear optimization problem including a priori information only on nonlinear input parameters. It is assumed that this information can be obtained from a training set, the distribution of a generating set, or linguistic information. To solve the problem, the Gauss–Newton method with linearization in the vicinity of the last estimate, asymptotic representations of the pseudo-inverse of perturbed matrices, and separable structures of models are used. The obtained algorithms have the following important properties: they do not require the selection of initial values of linearly entering parameters, which can lead to divergence, but, at the same time, it is not necessary to find partial derivatives of a projection matrix; they can be used in serial and batch processing; well-known algorithms are obtained from them as special cases, and a simulation has shown that the proposed algorithms can outperform the former in accuracy and convergence rate.