Reduction of parameters in fully connected layers of neural networks by low rank factorizations

Reduction of parameters in fully connected layers of neural networks by low rank factorizations

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y∈Rm is the output vector, x∈Rn is the input vector, and W∈Rm×n is a matrix of connection parameters, matrices Uij and Vij are defined and s...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Taylor, Graham, Sozubek, Serdar, Dalton, Barnaby, Courville, Vanessa
Format: Patent
Sprache:eng
Schlagworte:
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
PHYSICS
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y∈Rm is the output vector, x∈Rn is the input vector, and W∈Rm×n is a matrix of connection parameters, matrices Uij and Vij are defined and submatrices Wij are computed as the product of Uij and Vij, so that Wij=VijUij, and W is obtained by appending submatrices Wi,j.