LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking

Deep learning models, particularly those based on transformers, often employ numerous stacked structures, which possess identical architectures and perform similar functions. While effective, this stacking paradigm leads to a substantial increase in the number of parameters, posing challenges for pr...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-03
Hauptverfasser:	Li, Jialin, Nie, Qiang, Fu, Weifu, Lin, Yuhuan, Guangpin Tao, Liu, Yong, Wang, Chengjie
Format:	Artikel
Sprache:	eng
Schlagworte:	Decoders Mathematical models Modules Parameters
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!