A comprehensive, application-oriented study of catastrophic forgetting in DNNs
ICLR 2019 International Conference on Learning Representations We present a large-scale empirical study of catastrophic forgetting (CF) in modern Deep Neural Network (DNN) models that perform sequential (or: incremental) learning. A new experimental protocol is proposed that enforces typical constra...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | |
container_volume | |
creator | Pfülb, B Gepperth, A |
description | ICLR 2019 International Conference on Learning Representations We present a large-scale empirical study of catastrophic forgetting (CF) in
modern Deep Neural Network (DNN) models that perform sequential (or:
incremental) learning. A new experimental protocol is proposed that enforces
typical constraints encountered in application scenarios. As the investigation
is empirical, we evaluate CF behavior on the hitherto largest number of visual
classification datasets, from each of which we construct a representative
number of Sequential Learning Tasks (SLTs) in close alignment to previous works
on CF. Our results clearly indicate that there is no model that avoids CF for
all investigated datasets and SLTs under application conditions. We conclude
with a discussion of potential solutions and workarounds to CF, notably for the
EWC and IMM models. |
doi_str_mv | 10.48550/arxiv.1905.08101 |
format | Article |
fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1905_08101</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1905_08101</sourcerecordid><originalsourceid>FETCH-LOGICAL-a671-d8e38bf76db58031795dfc99b6f534cbc58fd70e480bfadeea785c2aba60f53a3</originalsourceid><addsrcrecordid>eNotz81OAyEUBWA2Lkz1AVzJAzgjyDAwy6b-Js246X5ygUtL0gJhsLFvb62uzuKcnOQj5I6zttNSskco3-HY8oHJlmnO-DUZl9SmQy64wziHIz5QyHkfLNSQYpNKwFjR0bl-uRNNnp4LmGtJeRcs9alssdYQtzRE-jyO8w258rCf8fY_F2Tz-rJZvTfrz7eP1XLdQK944zQKbbzqnZGaCa4G6bwdBtN7KTprrNTeKYadZsaDQwSlpX0CAz07L0AsyP3f7QU05RIOUE7TL2y6wMQPN7pKbw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A comprehensive, application-oriented study of catastrophic forgetting in DNNs</title><source>arXiv.org</source><creator>Pfülb, B ; Gepperth, A</creator><creatorcontrib>Pfülb, B ; Gepperth, A</creatorcontrib><description>ICLR 2019 International Conference on Learning Representations We present a large-scale empirical study of catastrophic forgetting (CF) in
modern Deep Neural Network (DNN) models that perform sequential (or:
incremental) learning. A new experimental protocol is proposed that enforces
typical constraints encountered in application scenarios. As the investigation
is empirical, we evaluate CF behavior on the hitherto largest number of visual
classification datasets, from each of which we construct a representative
number of Sequential Learning Tasks (SLTs) in close alignment to previous works
on CF. Our results clearly indicate that there is no model that avoids CF for
all investigated datasets and SLTs under application conditions. We conclude
with a discussion of potential solutions and workarounds to CF, notably for the
EWC and IMM models.</description><identifier>DOI: 10.48550/arxiv.1905.08101</identifier><language>eng</language><subject>Computer Science - Learning ; Statistics - Machine Learning</subject><creationdate>2019-05</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1905.08101$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1905.08101$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Pfülb, B</creatorcontrib><creatorcontrib>Gepperth, A</creatorcontrib><title>A comprehensive, application-oriented study of catastrophic forgetting in DNNs</title><description>ICLR 2019 International Conference on Learning Representations We present a large-scale empirical study of catastrophic forgetting (CF) in
modern Deep Neural Network (DNN) models that perform sequential (or:
incremental) learning. A new experimental protocol is proposed that enforces
typical constraints encountered in application scenarios. As the investigation
is empirical, we evaluate CF behavior on the hitherto largest number of visual
classification datasets, from each of which we construct a representative
number of Sequential Learning Tasks (SLTs) in close alignment to previous works
on CF. Our results clearly indicate that there is no model that avoids CF for
all investigated datasets and SLTs under application conditions. We conclude
with a discussion of potential solutions and workarounds to CF, notably for the
EWC and IMM models.</description><subject>Computer Science - Learning</subject><subject>Statistics - Machine Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotz81OAyEUBWA2Lkz1AVzJAzgjyDAwy6b-Js246X5ygUtL0gJhsLFvb62uzuKcnOQj5I6zttNSskco3-HY8oHJlmnO-DUZl9SmQy64wziHIz5QyHkfLNSQYpNKwFjR0bl-uRNNnp4LmGtJeRcs9alssdYQtzRE-jyO8w258rCf8fY_F2Tz-rJZvTfrz7eP1XLdQK944zQKbbzqnZGaCa4G6bwdBtN7KTprrNTeKYadZsaDQwSlpX0CAz07L0AsyP3f7QU05RIOUE7TL2y6wMQPN7pKbw</recordid><startdate>20190520</startdate><enddate>20190520</enddate><creator>Pfülb, B</creator><creator>Gepperth, A</creator><scope>AKY</scope><scope>EPD</scope><scope>GOX</scope></search><sort><creationdate>20190520</creationdate><title>A comprehensive, application-oriented study of catastrophic forgetting in DNNs</title><author>Pfülb, B ; Gepperth, A</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a671-d8e38bf76db58031795dfc99b6f534cbc58fd70e480bfadeea785c2aba60f53a3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Computer Science - Learning</topic><topic>Statistics - Machine Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Pfülb, B</creatorcontrib><creatorcontrib>Gepperth, A</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv Statistics</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Pfülb, B</au><au>Gepperth, A</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A comprehensive, application-oriented study of catastrophic forgetting in DNNs</atitle><date>2019-05-20</date><risdate>2019</risdate><abstract>ICLR 2019 International Conference on Learning Representations We present a large-scale empirical study of catastrophic forgetting (CF) in
modern Deep Neural Network (DNN) models that perform sequential (or:
incremental) learning. A new experimental protocol is proposed that enforces
typical constraints encountered in application scenarios. As the investigation
is empirical, we evaluate CF behavior on the hitherto largest number of visual
classification datasets, from each of which we construct a representative
number of Sequential Learning Tasks (SLTs) in close alignment to previous works
on CF. Our results clearly indicate that there is no model that avoids CF for
all investigated datasets and SLTs under application conditions. We conclude
with a discussion of potential solutions and workarounds to CF, notably for the
EWC and IMM models.</abstract><doi>10.48550/arxiv.1905.08101</doi><oa>free_for_read</oa></addata></record> |
fulltext | fulltext_linktorsrc |
identifier | DOI: 10.48550/arxiv.1905.08101 |
ispartof | |
issn | |
language | eng |
recordid | cdi_arxiv_primary_1905_08101 |
source | arXiv.org |
subjects | Computer Science - Learning Statistics - Machine Learning |
title | A comprehensive, application-oriented study of catastrophic forgetting in DNNs |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T18%3A30%3A09IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20comprehensive,%20application-oriented%20study%20of%20catastrophic%20forgetting%20in%20DNNs&rft.au=Pf%C3%BClb,%20B&rft.date=2019-05-20&rft_id=info:doi/10.48550/arxiv.1905.08101&rft_dat=%3Carxiv_GOX%3E1905_08101%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |