Attention-based Knowledge Distillation in Multi-attention Tasks: The Impact of a DCT-driven Loss

Knowledge Distillation (KD) is a strategy for the definition of a set of transferability gangways to improve the efficiency of Convolutional Neural Networks. Feature-based Knowledge Distillation is a subfield of KD that relies on intermediate network representations, either unaltered or depth-reduce...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2022-06
Hauptverfasser:	López-Cifuentes, Alejandro, Escudero-Viñolo, Marcos, Bescós, Jesús, SanMiguel, Juan C
Format:	Artikel
Sprache:	eng
Schlagworte:	Artificial neural networks Distillation Recognition Two dimensional analysis
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!