If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks

•Recent work has shown that dropout limits the depth to which information can propagate through a neural network.•We investigate the effect of initialisation on training speed and generalisation within this depth limit.•We ask specifically: if dropout limits depth, does initialising critically still...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	Pattern recognition letters 2020-10, Vol.138, p.95-105
Hauptverfasser:	Pretorius, Arnu, van Biljon, Elan, van Niekerk, Benjamin, Eloff, Ryan, Reynard, Matthew, James, Steve, Rosman, Benjamin, Kamper, Herman, Kroon, Steve
Format:	Artikel
Sprache:	eng
Schlagworte:	Critical initialisation Neural networks Randomised control trial Signal propagation Statistical analysis Statistics Training
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!