Deep belief echo-state network and its application to time series prediction
•A novel deep neural network architecture called DBEN is presented for time series prediction.•The deep model contains a pre-train network and a novel regression layer.•The ESN methodology is introduced in the regression layer to perform a local fine-tuning.•Experimental results show considerable im...
Gespeichert in:
Veröffentlicht in: | Knowledge-based systems 2017-08, Vol.130, p.17-29 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | •A novel deep neural network architecture called DBEN is presented for time series prediction.•The deep model contains a pre-train network and a novel regression layer.•The ESN methodology is introduced in the regression layer to perform a local fine-tuning.•Experimental results show considerable improvements in the prediction performance, learning speed and STM.
Deep belief network (DBN) has attracted many attentions in time series prediction. However, the DBN-based methods fail to provide favorable prediction results due to the congenital defects of the back-propagation method, such as slow convergence and local optimum. To address the problems, we propose a deep belief echo-state network (DBEN) for time series prediction. In the new architecture, DBN is employed for feature learning in an unsupervised fashion, which can effectively extract hierarchical data features. An innovative regression layer, embedding an echo-state learning mechanism instead of the traditional back-propagation method, is built on top of DBN for supervised prediction. To our best knowledge, this is the first paper that applies the echo state network methodology to deep learning. The resulted model, combining the merits of DBN and ESN, provides a more robust alternative to conventional deep neural networks for the superior prediction capacity. Extensive experimental results show that our DBEN can achieve a significant enhancement in the prediction performance, learning speed, and short-term memory capacity. |
---|---|
ISSN: | 0950-7051 1872-7409 |
DOI: | 10.1016/j.knosys.2017.05.022 |