DiagViB-6: A Diagnostic Benchmark Suite for Vision Models in the Presence of Shortcut and Generalization Opportunities
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10655-10664 Common deep neural networks (DNNs) for image classification have been shown to rely on shortcut opportunities (SO) in the form of predictive and easy-to-represent visual factors. This is known as sh...
Gespeichert in:
Hauptverfasser: | , , , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Proceedings of the IEEE/CVF International Conference on Computer
Vision (ICCV), 2021, pp. 10655-10664 Common deep neural networks (DNNs) for image classification have been shown
to rely on shortcut opportunities (SO) in the form of predictive and
easy-to-represent visual factors. This is known as shortcut learning and leads
to impaired generalization. In this work, we show that common DNNs also suffer
from shortcut learning when predicting only basic visual object factors of
variation (FoV) such as shape, color, or texture. We argue that besides
shortcut opportunities, generalization opportunities (GO) are also an inherent
part of real-world vision data and arise from partial independence between
predicted classes and FoVs. We also argue that it is necessary for DNNs to
exploit GO to overcome shortcut learning. Our core contribution is to introduce
the Diagnostic Vision Benchmark suite DiagViB-6, which includes datasets and
metrics to study a network's shortcut vulnerability and generalization
capability for six independent FoV. In particular, DiagViB-6 allows controlling
the type and degree of SO and GO in a dataset. We benchmark a wide range of
popular vision architectures and show that they can exploit GO only to a
limited extent. |
---|---|
DOI: | 10.48550/arxiv.2108.05779 |