Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience
As deep learning systems are scaled up to many billions of parameters, relating their internal structure to external behaviors becomes very challenging. Although daunting, this problem is not new: Neuroscientists and cognitive scientists have accumulated decades of experience analyzing a particularl...
Gespeichert in:
Hauptverfasser: | , , , , , , , , , , , , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | As deep learning systems are scaled up to many billions of parameters,
relating their internal structure to external behaviors becomes very
challenging. Although daunting, this problem is not new: Neuroscientists and
cognitive scientists have accumulated decades of experience analyzing a
particularly complex system - the brain. In this work, we argue that
interpreting both biological and artificial neural systems requires analyzing
those systems at multiple levels of analysis, with different analytic tools for
each level. We first lay out a joint grand challenge among scientists who study
the brain and who study artificial neural networks: understanding how
distributed neural mechanisms give rise to complex cognition and behavior. We
then present a series of analytical tools that can be used to analyze
biological and artificial neural systems, organizing those tools according to
Marr's three levels of analysis: computation/behavior,
algorithm/representation, and implementation. Overall, the multilevel
interpretability framework provides a principled way to tackle neural system
complexity; links structure, computation, and behavior; clarifies assumptions
and research priorities at each level; and paves the way toward a unified
effort for understanding intelligent systems, may they be biological or
artificial. |
---|---|
DOI: | 10.48550/arxiv.2408.12664 |