Analyzing concept drift and shift from sample data

Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping —the description and analysis of instances of concept drift or shift. We argue that concept drift ma...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Data mining and knowledge discovery 2018-09, Vol.32 (5), p.1179-1199
Hauptverfasser: Webb, Geoffrey I., Lee, Loong Kuan, Goethals, Bart, Petitjean, François
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 1199
container_issue 5
container_start_page 1179
container_title Data mining and knowledge discovery
container_volume 32
creator Webb, Geoffrey I.
Lee, Loong Kuan
Goethals, Bart
Petitjean, François
description Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping —the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.
doi_str_mv 10.1007/s10618-018-0554-1
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2015536658</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2015536658</sourcerecordid><originalsourceid>FETCH-LOGICAL-c316t-71062cc102fac09a479155528cc43ba34b2d48d83a240c868d2725d9e010c0843</originalsourceid><addsrcrecordid>eNp1UEtLAzEYDKJgrf4Abwueo9-Xx272WIpaoeBFwVtIk2zdsi-T7aH-erOs4MnDMHOYGYYh5BbhHgGKh4iQo6IwQUpB8YwsUBacFjL_OE-aK0GlQrgkVzEeAEAyDgvCVp1pTt91t89s31k_jJkLdTVmpnNZ_JxUFfo2i6YdGp85M5prclGZJvqbX16S96fHt_WGbl-fX9arLbUc85EWaRGzFoFVxkJpRFGilJIpawXfGS52zAnlFDdMgFW5cqxg0pUeECwowZfkbu4dQv919HHUh_4Y0tyoGaQqnudSJRfOLhv6GIOv9BDq1oSTRtDTNXq-RsOEdI3GlGFzJiZvt_fhr_n_0A8F22P0</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2015536658</pqid></control><display><type>article</type><title>Analyzing concept drift and shift from sample data</title><source>SpringerLink Journals - AutoHoldings</source><creator>Webb, Geoffrey I. ; Lee, Loong Kuan ; Goethals, Bart ; Petitjean, François</creator><creatorcontrib>Webb, Geoffrey I. ; Lee, Loong Kuan ; Goethals, Bart ; Petitjean, François</creatorcontrib><description>Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping —the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.</description><identifier>ISSN: 1384-5810</identifier><identifier>EISSN: 1573-756X</identifier><identifier>DOI: 10.1007/s10618-018-0554-1</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Airline operations ; Artificial Intelligence ; Chemistry and Earth Sciences ; Computer Science ; Data mining ; Data Mining and Knowledge Discovery ; Drift ; Information Storage and Retrieval ; Journal Track of ECML PKDD 2018 ; Machine learning ; Mapping ; Physics ; Statistics for Engineering</subject><ispartof>Data mining and knowledge discovery, 2018-09, Vol.32 (5), p.1179-1199</ispartof><rights>The Author(s) 2018</rights><rights>Data Mining and Knowledge Discovery is a copyright of Springer, (2018). All Rights Reserved.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c316t-71062cc102fac09a479155528cc43ba34b2d48d83a240c868d2725d9e010c0843</citedby><cites>FETCH-LOGICAL-c316t-71062cc102fac09a479155528cc43ba34b2d48d83a240c868d2725d9e010c0843</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s10618-018-0554-1$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s10618-018-0554-1$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,780,784,27923,27924,41487,42556,51318</link.rule.ids></links><search><creatorcontrib>Webb, Geoffrey I.</creatorcontrib><creatorcontrib>Lee, Loong Kuan</creatorcontrib><creatorcontrib>Goethals, Bart</creatorcontrib><creatorcontrib>Petitjean, François</creatorcontrib><title>Analyzing concept drift and shift from sample data</title><title>Data mining and knowledge discovery</title><addtitle>Data Min Knowl Disc</addtitle><description>Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping —the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.</description><subject>Airline operations</subject><subject>Artificial Intelligence</subject><subject>Chemistry and Earth Sciences</subject><subject>Computer Science</subject><subject>Data mining</subject><subject>Data Mining and Knowledge Discovery</subject><subject>Drift</subject><subject>Information Storage and Retrieval</subject><subject>Journal Track of ECML PKDD 2018</subject><subject>Machine learning</subject><subject>Mapping</subject><subject>Physics</subject><subject>Statistics for Engineering</subject><issn>1384-5810</issn><issn>1573-756X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2018</creationdate><recordtype>article</recordtype><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp1UEtLAzEYDKJgrf4Abwueo9-Xx272WIpaoeBFwVtIk2zdsi-T7aH-erOs4MnDMHOYGYYh5BbhHgGKh4iQo6IwQUpB8YwsUBacFjL_OE-aK0GlQrgkVzEeAEAyDgvCVp1pTt91t89s31k_jJkLdTVmpnNZ_JxUFfo2i6YdGp85M5prclGZJvqbX16S96fHt_WGbl-fX9arLbUc85EWaRGzFoFVxkJpRFGilJIpawXfGS52zAnlFDdMgFW5cqxg0pUeECwowZfkbu4dQv919HHUh_4Y0tyoGaQqnudSJRfOLhv6GIOv9BDq1oSTRtDTNXq-RsOEdI3GlGFzJiZvt_fhr_n_0A8F22P0</recordid><startdate>20180901</startdate><enddate>20180901</enddate><creator>Webb, Geoffrey I.</creator><creator>Lee, Loong Kuan</creator><creator>Goethals, Bart</creator><creator>Petitjean, François</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>8AL</scope><scope>8AO</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>L.-</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M0N</scope><scope>M2O</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope></search><sort><creationdate>20180901</creationdate><title>Analyzing concept drift and shift from sample data</title><author>Webb, Geoffrey I. ; Lee, Loong Kuan ; Goethals, Bart ; Petitjean, François</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c316t-71062cc102fac09a479155528cc43ba34b2d48d83a240c868d2725d9e010c0843</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2018</creationdate><topic>Airline operations</topic><topic>Artificial Intelligence</topic><topic>Chemistry and Earth Sciences</topic><topic>Computer Science</topic><topic>Data mining</topic><topic>Data Mining and Knowledge Discovery</topic><topic>Drift</topic><topic>Information Storage and Retrieval</topic><topic>Journal Track of ECML PKDD 2018</topic><topic>Machine learning</topic><topic>Mapping</topic><topic>Physics</topic><topic>Statistics for Engineering</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Webb, Geoffrey I.</creatorcontrib><creatorcontrib>Lee, Loong Kuan</creatorcontrib><creatorcontrib>Goethals, Bart</creatorcontrib><creatorcontrib>Petitjean, François</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Computing Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection (ProQuest)</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>ABI/INFORM Professional Advanced</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Computing Database</collection><collection>Research Library</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><jtitle>Data mining and knowledge discovery</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Webb, Geoffrey I.</au><au>Lee, Loong Kuan</au><au>Goethals, Bart</au><au>Petitjean, François</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Analyzing concept drift and shift from sample data</atitle><jtitle>Data mining and knowledge discovery</jtitle><stitle>Data Min Knowl Disc</stitle><date>2018-09-01</date><risdate>2018</risdate><volume>32</volume><issue>5</issue><spage>1179</spage><epage>1199</epage><pages>1179-1199</pages><issn>1384-5810</issn><eissn>1573-756X</eissn><abstract>Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping —the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s10618-018-0554-1</doi><tpages>21</tpages></addata></record>
fulltext fulltext
identifier ISSN: 1384-5810
ispartof Data mining and knowledge discovery, 2018-09, Vol.32 (5), p.1179-1199
issn 1384-5810
1573-756X
language eng
recordid cdi_proquest_journals_2015536658
source SpringerLink Journals - AutoHoldings
subjects Airline operations
Artificial Intelligence
Chemistry and Earth Sciences
Computer Science
Data mining
Data Mining and Knowledge Discovery
Drift
Information Storage and Retrieval
Journal Track of ECML PKDD 2018
Machine learning
Mapping
Physics
Statistics for Engineering
title Analyzing concept drift and shift from sample data
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-08T07%3A43%3A54IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Analyzing%20concept%20drift%20and%20shift%20from%20sample%20data&rft.jtitle=Data%20mining%20and%20knowledge%20discovery&rft.au=Webb,%20Geoffrey%20I.&rft.date=2018-09-01&rft.volume=32&rft.issue=5&rft.spage=1179&rft.epage=1199&rft.pages=1179-1199&rft.issn=1384-5810&rft.eissn=1573-756X&rft_id=info:doi/10.1007/s10618-018-0554-1&rft_dat=%3Cproquest_cross%3E2015536658%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2015536658&rft_id=info:pmid/&rfr_iscdi=true