Data Contamination Report from the 2024 CONDA Shared Task
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-training corpora used to train large scale models, compromising eval...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Sainz, Oscar García-Ferrero, Iker Jacovi, Alon Campos, Jon Ander Elazar, Yanai Agirre, Eneko Goldberg, Yoav Chen, Wei-Lin Chim, Jenny Choshen, Leshem D'Amico-Wong, Luca Dell, Melissa Fan, Run-Ze Golchin, Shahriar Li, Yucheng Liu, Pengfei Pahwa, Bhavish Prabhu, Ameya Sharma, Suryansh Silcock, Emily Solonko, Kateryna Stap, David Surdeanu, Mihai Tseng, Yu-Min Udandarao, Vishaal Wang, Zengzhi Xu, Ruijie Yang, Jinglin |
| description | The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community. |
| doi_str_mv | 10.48550/arxiv.2407.21530 |
| format | Article |
| fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2407_21530</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2407_21530</sourcerecordid><originalsourceid>FETCH-arxiv_primary_2407_215303</originalsourceid><addsrcrecordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjEw1zMyNDU24GSwdEksSVRwzs8rSczNzEssyczPUwhKLcgvKlFIK8rPVSjJSFUwMjAyUXD293NxVAjOSCxKTVEISSzO5mFgTUvMKU7lhdLcDPJuriHOHrpgS-ILijJzE4sq40GWxYMtMyasAgB_CzIL</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Data Contamination Report from the 2024 CONDA Shared Task</title><source>arXiv.org</source><creator>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Campos, Jon Ander ; Elazar, Yanai ; Agirre, Eneko ; Goldberg, Yoav ; Chen, Wei-Lin ; Chim, Jenny ; Choshen, Leshem ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Tseng, Yu-Min ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</creator><creatorcontrib>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Campos, Jon Ander ; Elazar, Yanai ; Agirre, Eneko ; Goldberg, Yoav ; Chen, Wei-Lin ; Chim, Jenny ; Choshen, Leshem ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Tseng, Yu-Min ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</creatorcontrib><description>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community.</description><identifier>DOI: 10.48550/arxiv.2407.21530</identifier><language>eng</language><subject>Computer Science - Computation and Language ; Computer Science - Learning</subject><creationdate>2024-07</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2407.21530$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2407.21530$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Sainz, Oscar</creatorcontrib><creatorcontrib>García-Ferrero, Iker</creatorcontrib><creatorcontrib>Jacovi, Alon</creatorcontrib><creatorcontrib>Campos, Jon Ander</creatorcontrib><creatorcontrib>Elazar, Yanai</creatorcontrib><creatorcontrib>Agirre, Eneko</creatorcontrib><creatorcontrib>Goldberg, Yoav</creatorcontrib><creatorcontrib>Chen, Wei-Lin</creatorcontrib><creatorcontrib>Chim, Jenny</creatorcontrib><creatorcontrib>Choshen, Leshem</creatorcontrib><creatorcontrib>D'Amico-Wong, Luca</creatorcontrib><creatorcontrib>Dell, Melissa</creatorcontrib><creatorcontrib>Fan, Run-Ze</creatorcontrib><creatorcontrib>Golchin, Shahriar</creatorcontrib><creatorcontrib>Li, Yucheng</creatorcontrib><creatorcontrib>Liu, Pengfei</creatorcontrib><creatorcontrib>Pahwa, Bhavish</creatorcontrib><creatorcontrib>Prabhu, Ameya</creatorcontrib><creatorcontrib>Sharma, Suryansh</creatorcontrib><creatorcontrib>Silcock, Emily</creatorcontrib><creatorcontrib>Solonko, Kateryna</creatorcontrib><creatorcontrib>Stap, David</creatorcontrib><creatorcontrib>Surdeanu, Mihai</creatorcontrib><creatorcontrib>Tseng, Yu-Min</creatorcontrib><creatorcontrib>Udandarao, Vishaal</creatorcontrib><creatorcontrib>Wang, Zengzhi</creatorcontrib><creatorcontrib>Xu, Ruijie</creatorcontrib><creatorcontrib>Yang, Jinglin</creatorcontrib><title>Data Contamination Report from the 2024 CONDA Shared Task</title><description>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community.</description><subject>Computer Science - Computation and Language</subject><subject>Computer Science - Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjEw1zMyNDU24GSwdEksSVRwzs8rSczNzEssyczPUwhKLcgvKlFIK8rPVSjJSFUwMjAyUXD293NxVAjOSCxKTVEISSzO5mFgTUvMKU7lhdLcDPJuriHOHrpgS-ILijJzE4sq40GWxYMtMyasAgB_CzIL</recordid><startdate>20240731</startdate><enddate>20240731</enddate><creator>Sainz, Oscar</creator><creator>García-Ferrero, Iker</creator><creator>Jacovi, Alon</creator><creator>Campos, Jon Ander</creator><creator>Elazar, Yanai</creator><creator>Agirre, Eneko</creator><creator>Goldberg, Yoav</creator><creator>Chen, Wei-Lin</creator><creator>Chim, Jenny</creator><creator>Choshen, Leshem</creator><creator>D'Amico-Wong, Luca</creator><creator>Dell, Melissa</creator><creator>Fan, Run-Ze</creator><creator>Golchin, Shahriar</creator><creator>Li, Yucheng</creator><creator>Liu, Pengfei</creator><creator>Pahwa, Bhavish</creator><creator>Prabhu, Ameya</creator><creator>Sharma, Suryansh</creator><creator>Silcock, Emily</creator><creator>Solonko, Kateryna</creator><creator>Stap, David</creator><creator>Surdeanu, Mihai</creator><creator>Tseng, Yu-Min</creator><creator>Udandarao, Vishaal</creator><creator>Wang, Zengzhi</creator><creator>Xu, Ruijie</creator><creator>Yang, Jinglin</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240731</creationdate><title>Data Contamination Report from the 2024 CONDA Shared Task</title><author>Sainz, Oscar ; García-Ferrero, Iker ; Jacovi, Alon ; Campos, Jon Ander ; Elazar, Yanai ; Agirre, Eneko ; Goldberg, Yoav ; Chen, Wei-Lin ; Chim, Jenny ; Choshen, Leshem ; D'Amico-Wong, Luca ; Dell, Melissa ; Fan, Run-Ze ; Golchin, Shahriar ; Li, Yucheng ; Liu, Pengfei ; Pahwa, Bhavish ; Prabhu, Ameya ; Sharma, Suryansh ; Silcock, Emily ; Solonko, Kateryna ; Stap, David ; Surdeanu, Mihai ; Tseng, Yu-Min ; Udandarao, Vishaal ; Wang, Zengzhi ; Xu, Ruijie ; Yang, Jinglin</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-arxiv_primary_2407_215303</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Computation and Language</topic><topic>Computer Science - Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Sainz, Oscar</creatorcontrib><creatorcontrib>García-Ferrero, Iker</creatorcontrib><creatorcontrib>Jacovi, Alon</creatorcontrib><creatorcontrib>Campos, Jon Ander</creatorcontrib><creatorcontrib>Elazar, Yanai</creatorcontrib><creatorcontrib>Agirre, Eneko</creatorcontrib><creatorcontrib>Goldberg, Yoav</creatorcontrib><creatorcontrib>Chen, Wei-Lin</creatorcontrib><creatorcontrib>Chim, Jenny</creatorcontrib><creatorcontrib>Choshen, Leshem</creatorcontrib><creatorcontrib>D'Amico-Wong, Luca</creatorcontrib><creatorcontrib>Dell, Melissa</creatorcontrib><creatorcontrib>Fan, Run-Ze</creatorcontrib><creatorcontrib>Golchin, Shahriar</creatorcontrib><creatorcontrib>Li, Yucheng</creatorcontrib><creatorcontrib>Liu, Pengfei</creatorcontrib><creatorcontrib>Pahwa, Bhavish</creatorcontrib><creatorcontrib>Prabhu, Ameya</creatorcontrib><creatorcontrib>Sharma, Suryansh</creatorcontrib><creatorcontrib>Silcock, Emily</creatorcontrib><creatorcontrib>Solonko, Kateryna</creatorcontrib><creatorcontrib>Stap, David</creatorcontrib><creatorcontrib>Surdeanu, Mihai</creatorcontrib><creatorcontrib>Tseng, Yu-Min</creatorcontrib><creatorcontrib>Udandarao, Vishaal</creatorcontrib><creatorcontrib>Wang, Zengzhi</creatorcontrib><creatorcontrib>Xu, Ruijie</creatorcontrib><creatorcontrib>Yang, Jinglin</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Sainz, Oscar</au><au>García-Ferrero, Iker</au><au>Jacovi, Alon</au><au>Campos, Jon Ander</au><au>Elazar, Yanai</au><au>Agirre, Eneko</au><au>Goldberg, Yoav</au><au>Chen, Wei-Lin</au><au>Chim, Jenny</au><au>Choshen, Leshem</au><au>D'Amico-Wong, Luca</au><au>Dell, Melissa</au><au>Fan, Run-Ze</au><au>Golchin, Shahriar</au><au>Li, Yucheng</au><au>Liu, Pengfei</au><au>Pahwa, Bhavish</au><au>Prabhu, Ameya</au><au>Sharma, Suryansh</au><au>Silcock, Emily</au><au>Solonko, Kateryna</au><au>Stap, David</au><au>Surdeanu, Mihai</au><au>Tseng, Yu-Min</au><au>Udandarao, Vishaal</au><au>Wang, Zengzhi</au><au>Xu, Ruijie</au><au>Yang, Jinglin</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Data Contamination Report from the 2024 CONDA Shared Task</atitle><date>2024-07-31</date><risdate>2024</risdate><abstract>The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant
aspects of data contamination in natural language processing, where data
contamination is understood as situations where evaluation data is included in
pre-training corpora used to train large scale models, compromising evaluation
results. The workshop fostered a shared task to collect evidence on data
contamination in current available datasets and models. The goal of the shared
task and associated database is to assist the community in understanding the
extent of the problem and to assist researchers in avoiding reporting
evaluation results on known contaminated resources. The shared task provides a
structured, centralized public database for the collection of contamination
evidence, open to contributions from the community via GitHub pool requests.
This first compilation paper is based on 566 reported entries over 91
contaminated sources from a total of 23 contributors. The details of the
individual contamination events are available in the platform. The platform
continues to be online, open to contributions from the community.</abstract><doi>10.48550/arxiv.2407.21530</doi><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | DOI: 10.48550/arxiv.2407.21530 |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_arxiv_primary_2407_21530 |
| source | arXiv.org |
| subjects | Computer Science - Computation and Language Computer Science - Learning |
| title | Data Contamination Report from the 2024 CONDA Shared Task |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-19T14%3A16%3A11IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Data%20Contamination%20Report%20from%20the%202024%20CONDA%20Shared%20Task&rft.au=Sainz,%20Oscar&rft.date=2024-07-31&rft_id=info:doi/10.48550/arxiv.2407.21530&rft_dat=%3Carxiv_GOX%3E2407_21530%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |