A fast learning algorithm for time-delay neural networks

To solve the long training time in Waibel’s time-delay neural networks (TDNNs) for phoneme recognition, several improved fast learning methods are put forward: (1) by a combination between the unsupervised Oja’s rule learning method with the similar error backpropagation (BP) algorithm for initial weights training; (2) by improving of the error energy function with weights update according to the output error size; (3) by changing of BP error from along layers to frames and using the averaged overlapping part of frame-shift delta values in the weights of the bottom layer; (4) by training of the data from a small to large number of samples gradually; and (5) by using the optimal modular neural networks (OMNNs) with tree structure for multiple phonemes. Our experimental results indicate that the convergence speed is accelerated with orders of magnitude and in most cases the error function descends monotonically while the network complexity increases less and the recognition rates are almost the same among the different comparative experiments.