A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Forgetting refers to the loss or deterioration of previously acquired knowledge. While existing surveys on forgetting have primarily focused on continual learning, forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Forgetting manifests in research f...
Gespeichert in:
Hauptverfasser: | , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Forgetting refers to the loss or deterioration of previously acquired
knowledge. While existing surveys on forgetting have primarily focused on
continual learning, forgetting is a prevalent phenomenon observed in various
other research domains within deep learning. Forgetting manifests in research
fields such as generative models due to generator shifts, and federated
learning due to heterogeneous data distributions across clients. Addressing
forgetting encompasses several challenges, including balancing the retention of
old task knowledge with fast learning of new task, managing task interference
with conflicting goals, and preventing privacy leakage, etc. Moreover, most
existing surveys on continual learning implicitly assume that forgetting is
always harmful. In contrast, our survey argues that forgetting is a
double-edged sword and can be beneficial and desirable in certain cases, such
as privacy-preserving scenarios. By exploring forgetting in a broader context,
we present a more nuanced understanding of this phenomenon and highlight its
potential advantages. Through this comprehensive survey, we aspire to uncover
potential solutions by drawing upon ideas and approaches from various fields
that have dealt with forgetting. By examining forgetting beyond its
conventional boundaries, we hope to encourage the development of novel
strategies for mitigating, harnessing, or even embracing forgetting in real
applications. A comprehensive list of papers about forgetting in various
research fields is available at
\url{https://github.com/EnnengYang/Awesome-Forgetting-in-Deep-Learning}. |
---|---|
DOI: | 10.48550/arxiv.2307.09218 |