Human-inspired Explanations for Vision Transformers and Convolutional Neural Networks

Human-inspired Explanations for Vision Transformers and Convolutional Neural Networks

We introduce Foveation-based Explanations (FovEx), a novel human-inspired visual explainability (XAI) method for Deep Neural Networks. Our method achieves state-of-the-art performance on both transformer (on 4 out of 5 metrics) and convolutional models (on 3 out of 5 metrics), demonstrating its vers...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org 2024-08
Hauptverfasser: Mahadev Prasad Panda, Tiezzi, Matteo, Vilas, Martina, Roig, Gemma, Eskofier, Bjoern M, Zanca, Dario
Format: Artikel
Sprache:eng
Schlagworte:
Artificial intelligence
Artificial neural networks
Machine learning
Performance evaluation
Perturbation
State-of-the-art reviews
Vision
Visual aspects
Visual tasks
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We introduce Foveation-based Explanations (FovEx), a novel human-inspired visual explainability (XAI) method for Deep Neural Networks. Our method achieves state-of-the-art performance on both transformer (on 4 out of 5 metrics) and convolutional models (on 3 out of 5 metrics), demonstrating its versatility. Furthermore, we show the alignment between the explanation map produced by FovEx and human gaze patterns (+14\% in NSS compared to RISE, +203\% in NSS compared to gradCAM), enhancing our confidence in FovEx's ability to close the interpretation gap between humans and machines.
ISSN:2331-8422