Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison

COVID-19 has disrupted higher education globally, and there is scarce information about the “learning loss” in university students throughout this crisis. The goal of the study was to compare scores in a large-scale knowledge diagnostic exam applied to students admitted to the university, before and...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Educational assessment, evaluation and accountability evaluation and accountability, 2023-11, Vol.35 (4), p.503-523
Hauptverfasser: Sánchez-Mendiola, Melchor, Manzano-Patiño, Abigail P., García-Minjares, Manuel, Casanova, Enrique Buzo, Herrera Penilla, Careli J., Goytia-Rodríguez, Katyna, Martínez-González, Adrián
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 523
container_issue 4
container_start_page 503
container_title Educational assessment, evaluation and accountability
container_volume 35
creator Sánchez-Mendiola, Melchor
Manzano-Patiño, Abigail P.
García-Minjares, Manuel
Casanova, Enrique Buzo
Herrera Penilla, Careli J.
Goytia-Rodríguez, Katyna
Martínez-González, Adrián
description COVID-19 has disrupted higher education globally, and there is scarce information about the “learning loss” in university students throughout this crisis. The goal of the study was to compare scores in a large-scale knowledge diagnostic exam applied to students admitted to the university, before and during the pandemic. Research design was quasi-experimental with static group comparisons, taking advantage of the pandemic “natural experiment,” to assess knowledge in students admitted to the National Autonomous University of Mexico. Four student cohorts were analyzed: 2017 and 2018 (prepandemic, paper-and-pencil exams), 2020 and 2021 (transpandemic, online exams). The same instruments were applied in each pair of cohorts (2017–2021; 2018–2020) to decrease instrumentation threat. Propensity score matching was used to create balanced comparable groups. 35,584 matched students from each of the 2018 and 2020 cohorts were compared and 31,574 matched students from each of the 2017–2021 cohorts. Reliability and point biserial correlation coefficients were higher in the transpandemic online applications. Knowledge scores were 2.3 to 7.1% higher in the transpandemic assessments, Spanish scores in the 2018–2020 comparison were 1.3% lower, and English results in 2021 were 7.1% lower than in 2017. Before the pandemic, there was a 3.1% higher test performance in men; this gap decreased to 0.34% during the pandemic. There was no documented learning loss in this large student population, with an increase in knowledge in the pandemic cohorts. Some influence in scores due to the online testing modality cannot be ruled out. Longitudinal follow-up is required to continue evaluating the impact of the pandemic in learning.
doi_str_mv 10.1007/s11092-023-09410-9
format Article
fullrecord <record><control><sourceid>gale_proqu</sourceid><recordid>TN_cdi_gale_infotracmisc_A776853892</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A776853892</galeid><sourcerecordid>A776853892</sourcerecordid><originalsourceid>FETCH-LOGICAL-c430t-7d2202b775cdff09ff78e0c73af8347840d7326a6bc4f0a1ec15442eb82ab1763</originalsourceid><addsrcrecordid>eNp9kUtLLDEQRhtRUEf_gKuA62jl0Z3EnYgvGLibe9chk06GiJ1uUz3C_Hujo4gglyxShHMqVXxNc8bgggGoS2QMDKfABQUjGVCz1xwxrSTVHcD-V90addgcIz4BdMoYcdTEpSvrQNG750D65NZ5xDl54hAD4hDyTFImMRWc6Ta4QjY5vYaCad4SnDd9BfCKTCVQ4nJP5uIyTrUKQ23ix2FyJeGYT5qD6J4xnH7ei-bf3e3fmwe6_HP_eHO9pF4KmKnqOQe-Uqr1fYxgYlQ6gFfCRS2k0hJ6JXjnupWXERwLnrVS8rDS3K2Y6sSiOd_1ncr4sgk426dxU3L90nIDwuhWtvybWtetbcpxrHP7IaG310p1uhXavFMXv1D1fCw35hBTff8h8J3gy4hYQrRTSYMrW8vAvsdkdzHZGpP9iMmaKomdhBXO61C-J_6P9QYuZ5V5</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2903985452</pqid></control><display><type>article</type><title>Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison</title><source>SpringerLink Journals - AutoHoldings</source><creator>Sánchez-Mendiola, Melchor ; Manzano-Patiño, Abigail P. ; García-Minjares, Manuel ; Casanova, Enrique Buzo ; Herrera Penilla, Careli J. ; Goytia-Rodríguez, Katyna ; Martínez-González, Adrián</creator><creatorcontrib>Sánchez-Mendiola, Melchor ; Manzano-Patiño, Abigail P. ; García-Minjares, Manuel ; Casanova, Enrique Buzo ; Herrera Penilla, Careli J. ; Goytia-Rodríguez, Katyna ; Martínez-González, Adrián</creatorcontrib><description>COVID-19 has disrupted higher education globally, and there is scarce information about the “learning loss” in university students throughout this crisis. The goal of the study was to compare scores in a large-scale knowledge diagnostic exam applied to students admitted to the university, before and during the pandemic. Research design was quasi-experimental with static group comparisons, taking advantage of the pandemic “natural experiment,” to assess knowledge in students admitted to the National Autonomous University of Mexico. Four student cohorts were analyzed: 2017 and 2018 (prepandemic, paper-and-pencil exams), 2020 and 2021 (transpandemic, online exams). The same instruments were applied in each pair of cohorts (2017–2021; 2018–2020) to decrease instrumentation threat. Propensity score matching was used to create balanced comparable groups. 35,584 matched students from each of the 2018 and 2020 cohorts were compared and 31,574 matched students from each of the 2017–2021 cohorts. Reliability and point biserial correlation coefficients were higher in the transpandemic online applications. Knowledge scores were 2.3 to 7.1% higher in the transpandemic assessments, Spanish scores in the 2018–2020 comparison were 1.3% lower, and English results in 2021 were 7.1% lower than in 2017. Before the pandemic, there was a 3.1% higher test performance in men; this gap decreased to 0.34% during the pandemic. There was no documented learning loss in this large student population, with an increase in knowledge in the pandemic cohorts. Some influence in scores due to the online testing modality cannot be ruled out. Longitudinal follow-up is required to continue evaluating the impact of the pandemic in learning.</description><identifier>ISSN: 1874-8597</identifier><identifier>EISSN: 1874-8600</identifier><identifier>DOI: 10.1007/s11092-023-09410-9</identifier><language>eng</language><publisher>Dordrecht: Springer Netherlands</publisher><subject>Achievement Gains ; Analysis ; Assessment ; College admissions ; College Students ; Correlation ; Distance learning ; Education ; Educational evaluation ; Grouping (Instructional Purposes) ; Learning ; Pandemics ; Research Design ; Testing and Evaluation ; University students</subject><ispartof>Educational assessment, evaluation and accountability, 2023-11, Vol.35 (4), p.503-523</ispartof><rights>The Author(s) 2023</rights><rights>COPYRIGHT 2023 Springer</rights><rights>The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c430t-7d2202b775cdff09ff78e0c73af8347840d7326a6bc4f0a1ec15442eb82ab1763</citedby><cites>FETCH-LOGICAL-c430t-7d2202b775cdff09ff78e0c73af8347840d7326a6bc4f0a1ec15442eb82ab1763</cites><orcidid>0000-0002-9664-3208 ; 0000-0002-9535-5917 ; 0000-0001-7490-7826 ; 0000-0002-6947-7996 ; 0000-0002-5021-9639 ; 0000-0002-1678-2196 ; 0000-0003-0434-0654</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s11092-023-09410-9$$EPDF$$P50$$Gspringer$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s11092-023-09410-9$$EHTML$$P50$$Gspringer$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Sánchez-Mendiola, Melchor</creatorcontrib><creatorcontrib>Manzano-Patiño, Abigail P.</creatorcontrib><creatorcontrib>García-Minjares, Manuel</creatorcontrib><creatorcontrib>Casanova, Enrique Buzo</creatorcontrib><creatorcontrib>Herrera Penilla, Careli J.</creatorcontrib><creatorcontrib>Goytia-Rodríguez, Katyna</creatorcontrib><creatorcontrib>Martínez-González, Adrián</creatorcontrib><title>Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison</title><title>Educational assessment, evaluation and accountability</title><addtitle>Educ Asse Eval Acc</addtitle><description>COVID-19 has disrupted higher education globally, and there is scarce information about the “learning loss” in university students throughout this crisis. The goal of the study was to compare scores in a large-scale knowledge diagnostic exam applied to students admitted to the university, before and during the pandemic. Research design was quasi-experimental with static group comparisons, taking advantage of the pandemic “natural experiment,” to assess knowledge in students admitted to the National Autonomous University of Mexico. Four student cohorts were analyzed: 2017 and 2018 (prepandemic, paper-and-pencil exams), 2020 and 2021 (transpandemic, online exams). The same instruments were applied in each pair of cohorts (2017–2021; 2018–2020) to decrease instrumentation threat. Propensity score matching was used to create balanced comparable groups. 35,584 matched students from each of the 2018 and 2020 cohorts were compared and 31,574 matched students from each of the 2017–2021 cohorts. Reliability and point biserial correlation coefficients were higher in the transpandemic online applications. Knowledge scores were 2.3 to 7.1% higher in the transpandemic assessments, Spanish scores in the 2018–2020 comparison were 1.3% lower, and English results in 2021 were 7.1% lower than in 2017. Before the pandemic, there was a 3.1% higher test performance in men; this gap decreased to 0.34% during the pandemic. There was no documented learning loss in this large student population, with an increase in knowledge in the pandemic cohorts. Some influence in scores due to the online testing modality cannot be ruled out. Longitudinal follow-up is required to continue evaluating the impact of the pandemic in learning.</description><subject>Achievement Gains</subject><subject>Analysis</subject><subject>Assessment</subject><subject>College admissions</subject><subject>College Students</subject><subject>Correlation</subject><subject>Distance learning</subject><subject>Education</subject><subject>Educational evaluation</subject><subject>Grouping (Instructional Purposes)</subject><subject>Learning</subject><subject>Pandemics</subject><subject>Research Design</subject><subject>Testing and Evaluation</subject><subject>University students</subject><issn>1874-8597</issn><issn>1874-8600</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>C6C</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><recordid>eNp9kUtLLDEQRhtRUEf_gKuA62jl0Z3EnYgvGLibe9chk06GiJ1uUz3C_Hujo4gglyxShHMqVXxNc8bgggGoS2QMDKfABQUjGVCz1xwxrSTVHcD-V90addgcIz4BdMoYcdTEpSvrQNG750D65NZ5xDl54hAD4hDyTFImMRWc6Ta4QjY5vYaCad4SnDd9BfCKTCVQ4nJP5uIyTrUKQ23ix2FyJeGYT5qD6J4xnH7ei-bf3e3fmwe6_HP_eHO9pF4KmKnqOQe-Uqr1fYxgYlQ6gFfCRS2k0hJ6JXjnupWXERwLnrVS8rDS3K2Y6sSiOd_1ncr4sgk426dxU3L90nIDwuhWtvybWtetbcpxrHP7IaG310p1uhXavFMXv1D1fCw35hBTff8h8J3gy4hYQrRTSYMrW8vAvsdkdzHZGpP9iMmaKomdhBXO61C-J_6P9QYuZ5V5</recordid><startdate>20231101</startdate><enddate>20231101</enddate><creator>Sánchez-Mendiola, Melchor</creator><creator>Manzano-Patiño, Abigail P.</creator><creator>García-Minjares, Manuel</creator><creator>Casanova, Enrique Buzo</creator><creator>Herrera Penilla, Careli J.</creator><creator>Goytia-Rodríguez, Katyna</creator><creator>Martínez-González, Adrián</creator><general>Springer Netherlands</general><general>Springer</general><general>Springer Nature B.V</general><scope>C6C</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>0-V</scope><scope>3V.</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>88B</scope><scope>88G</scope><scope>8A4</scope><scope>8AO</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>8FL</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ALSLI</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>CCPQU</scope><scope>CJNVE</scope><scope>DWQXO</scope><scope>FRNLG</scope><scope>FYUFA</scope><scope>F~G</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>K60</scope><scope>K6~</scope><scope>L.-</scope><scope>M0C</scope><scope>M0P</scope><scope>M2M</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEDU</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PSYQQ</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0002-9664-3208</orcidid><orcidid>https://orcid.org/0000-0002-9535-5917</orcidid><orcidid>https://orcid.org/0000-0001-7490-7826</orcidid><orcidid>https://orcid.org/0000-0002-6947-7996</orcidid><orcidid>https://orcid.org/0000-0002-5021-9639</orcidid><orcidid>https://orcid.org/0000-0002-1678-2196</orcidid><orcidid>https://orcid.org/0000-0003-0434-0654</orcidid></search><sort><creationdate>20231101</creationdate><title>Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison</title><author>Sánchez-Mendiola, Melchor ; Manzano-Patiño, Abigail P. ; García-Minjares, Manuel ; Casanova, Enrique Buzo ; Herrera Penilla, Careli J. ; Goytia-Rodríguez, Katyna ; Martínez-González, Adrián</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c430t-7d2202b775cdff09ff78e0c73af8347840d7326a6bc4f0a1ec15442eb82ab1763</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Achievement Gains</topic><topic>Analysis</topic><topic>Assessment</topic><topic>College admissions</topic><topic>College Students</topic><topic>Correlation</topic><topic>Distance learning</topic><topic>Education</topic><topic>Educational evaluation</topic><topic>Grouping (Instructional Purposes)</topic><topic>Learning</topic><topic>Pandemics</topic><topic>Research Design</topic><topic>Testing and Evaluation</topic><topic>University students</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Sánchez-Mendiola, Melchor</creatorcontrib><creatorcontrib>Manzano-Patiño, Abigail P.</creatorcontrib><creatorcontrib>García-Minjares, Manuel</creatorcontrib><creatorcontrib>Casanova, Enrique Buzo</creatorcontrib><creatorcontrib>Herrera Penilla, Careli J.</creatorcontrib><creatorcontrib>Goytia-Rodríguez, Katyna</creatorcontrib><creatorcontrib>Martínez-González, Adrián</creatorcontrib><collection>Springer Nature OA Free Journals</collection><collection>CrossRef</collection><collection>ProQuest Social Sciences Premium Collection</collection><collection>ProQuest Central (Corporate)</collection><collection>Access via ABI/INFORM (ProQuest)</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Education Database (Alumni Edition)</collection><collection>Psychology Database (Alumni)</collection><collection>Education Periodicals</collection><collection>ProQuest Pharma Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Social Science Premium Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>ProQuest One Community College</collection><collection>Education Collection</collection><collection>ProQuest Central Korea</collection><collection>Business Premium Collection (Alumni)</collection><collection>Health Research Premium Collection</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ABI/INFORM Global</collection><collection>Education Database</collection><collection>Psychology Database</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Education</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest One Psychology</collection><collection>ProQuest Central Basic</collection><jtitle>Educational assessment, evaluation and accountability</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sánchez-Mendiola, Melchor</au><au>Manzano-Patiño, Abigail P.</au><au>García-Minjares, Manuel</au><au>Casanova, Enrique Buzo</au><au>Herrera Penilla, Careli J.</au><au>Goytia-Rodríguez, Katyna</au><au>Martínez-González, Adrián</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison</atitle><jtitle>Educational assessment, evaluation and accountability</jtitle><stitle>Educ Asse Eval Acc</stitle><date>2023-11-01</date><risdate>2023</risdate><volume>35</volume><issue>4</issue><spage>503</spage><epage>523</epage><pages>503-523</pages><issn>1874-8597</issn><eissn>1874-8600</eissn><abstract>COVID-19 has disrupted higher education globally, and there is scarce information about the “learning loss” in university students throughout this crisis. The goal of the study was to compare scores in a large-scale knowledge diagnostic exam applied to students admitted to the university, before and during the pandemic. Research design was quasi-experimental with static group comparisons, taking advantage of the pandemic “natural experiment,” to assess knowledge in students admitted to the National Autonomous University of Mexico. Four student cohorts were analyzed: 2017 and 2018 (prepandemic, paper-and-pencil exams), 2020 and 2021 (transpandemic, online exams). The same instruments were applied in each pair of cohorts (2017–2021; 2018–2020) to decrease instrumentation threat. Propensity score matching was used to create balanced comparable groups. 35,584 matched students from each of the 2018 and 2020 cohorts were compared and 31,574 matched students from each of the 2017–2021 cohorts. Reliability and point biserial correlation coefficients were higher in the transpandemic online applications. Knowledge scores were 2.3 to 7.1% higher in the transpandemic assessments, Spanish scores in the 2018–2020 comparison were 1.3% lower, and English results in 2021 were 7.1% lower than in 2017. Before the pandemic, there was a 3.1% higher test performance in men; this gap decreased to 0.34% during the pandemic. There was no documented learning loss in this large student population, with an increase in knowledge in the pandemic cohorts. Some influence in scores due to the online testing modality cannot be ruled out. Longitudinal follow-up is required to continue evaluating the impact of the pandemic in learning.</abstract><cop>Dordrecht</cop><pub>Springer Netherlands</pub><doi>10.1007/s11092-023-09410-9</doi><tpages>21</tpages><orcidid>https://orcid.org/0000-0002-9664-3208</orcidid><orcidid>https://orcid.org/0000-0002-9535-5917</orcidid><orcidid>https://orcid.org/0000-0001-7490-7826</orcidid><orcidid>https://orcid.org/0000-0002-6947-7996</orcidid><orcidid>https://orcid.org/0000-0002-5021-9639</orcidid><orcidid>https://orcid.org/0000-0002-1678-2196</orcidid><orcidid>https://orcid.org/0000-0003-0434-0654</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1874-8597
ispartof Educational assessment, evaluation and accountability, 2023-11, Vol.35 (4), p.503-523
issn 1874-8597
1874-8600
language eng
recordid cdi_gale_infotracmisc_A776853892
source SpringerLink Journals - AutoHoldings
subjects Achievement Gains
Analysis
Assessment
College admissions
College Students
Correlation
Distance learning
Education
Educational evaluation
Grouping (Instructional Purposes)
Learning
Pandemics
Research Design
Testing and Evaluation
University students
title Large-scale diagnostic assessment in first-year university students: pre- and transpandemic comparison
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T06%3A23%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_proqu&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Large-scale%20diagnostic%20assessment%20in%20first-year%20university%20students:%20pre-%20and%20transpandemic%20comparison&rft.jtitle=Educational%20assessment,%20evaluation%20and%20accountability&rft.au=S%C3%A1nchez-Mendiola,%20Melchor&rft.date=2023-11-01&rft.volume=35&rft.issue=4&rft.spage=503&rft.epage=523&rft.pages=503-523&rft.issn=1874-8597&rft.eissn=1874-8600&rft_id=info:doi/10.1007/s11092-023-09410-9&rft_dat=%3Cgale_proqu%3EA776853892%3C/gale_proqu%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2903985452&rft_id=info:pmid/&rft_galeid=A776853892&rfr_iscdi=true