Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers

Vision transformers using self-attention or its proposed alternatives have demonstrated promising results in many image related tasks. However, the underpinning inductive bias of attention is not well understood. To address this issue, this paper analyzes attention through the lens of convex duality...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2022-05
Hauptverfasser:	Sahiner, Arda, Ergen, Tolga, Batu Ozturkler, Pauly, John, Mardani, Morteza, Pilanci, Mert
Format:	Artikel
Sprache:	eng
Schlagworte:	Bias Regularization
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!