Data practices in quality evaluation and assessment: Two universities at a glance

As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enha...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Higher education quarterly 2023-01, Vol.77 (1), p.7-26
Hauptverfasser: Raffaghelli, Juliana E., Grion, Valentina, Rossi, Marina
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 26
container_issue 1
container_start_page 7
container_title Higher education quarterly
container_volume 77
creator Raffaghelli, Juliana E.
Grion, Valentina
Rossi, Marina
description As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture. A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instit
doi_str_mv 10.1111/hequ.12361
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2782662444</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1362695</ericid><sourcerecordid>2766923298</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</originalsourceid><addsrcrecordid>eNp9kM1LAzEQxYMoWKsX70LAiwhb87XZjTep1SoFKbTnMGZTTdnutsluS_97U1c8eHAOM4f3m8fjIXRJyYDGufu0m3ZAGZf0CPWokHnCWMaPUY-olCYpY-IUnYWwJIRQykQPTR-hAbz2YBpnbMCuwpsWStfssd1C2ULj6gpDVWAIwYawslVzj2e7GreV21ofXOPiGzQY8EcJlbHn6GQBZbAXP7eP5k-j2XCcTN6eX4YPk8RwxWiSwwKEjCvLTSYLAUCJKdKikPw9MyZTKeGWFyktVE4AMsgWDMCkuVBcMGJ5H910vmtfb1obGr1ywdgyhrB1GzTLciYlE0JE9PoPuqxbX8V0kZJSMc5UHqnbjjK-DsHbhV57twK_15ToQ7v60K7-bjfCVx1svTO_4OiVcsmkSqNOO33nSrv_x0mPR9N55_kFn_uGIw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2766923298</pqid></control><display><type>article</type><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><source>PAIS Index</source><source>Access via Wiley Online Library</source><source>EBSCOhost Education Source</source><creator>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</creator><creatorcontrib>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</creatorcontrib><description>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture. A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</description><identifier>ISSN: 0951-5224</identifier><identifier>EISSN: 1468-2273</identifier><identifier>DOI: 10.1111/hequ.12361</identifier><language>eng</language><publisher>Oxford: Wiley</publisher><subject>Colleges &amp; universities ; Corporate culture ; Data analysis ; data cultures ; data practices ; Data quality ; Data Use ; Debates ; Educational Assessment ; Educational evaluation ; Educational Quality ; Evaluation ; Faculty Development ; Higher Education ; Organizational culture ; Quality of education ; Statistical inference ; Statistics ; Survey ; Teacher Effectiveness ; Teacher Evaluation ; Teaching ; Universities</subject><ispartof>Higher education quarterly, 2023-01, Vol.77 (1), p.7-26</ispartof><rights>2021 The Authors. published by John Wiley &amp; Sons Ltd.</rights><rights>2021. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</citedby><cites>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</cites><orcidid>0000-0002-8753-6478 ; 0000-0002-2051-1313 ; 0000-0002-5115-8196</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Fhequ.12361$$EPDF$$P50$$Gwiley$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Fhequ.12361$$EHTML$$P50$$Gwiley$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,1417,27866,27924,27925,45574,45575</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1362695$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Raffaghelli, Juliana E.</creatorcontrib><creatorcontrib>Grion, Valentina</creatorcontrib><creatorcontrib>Rossi, Marina</creatorcontrib><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><title>Higher education quarterly</title><description>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture. A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</description><subject>Colleges &amp; universities</subject><subject>Corporate culture</subject><subject>Data analysis</subject><subject>data cultures</subject><subject>data practices</subject><subject>Data quality</subject><subject>Data Use</subject><subject>Debates</subject><subject>Educational Assessment</subject><subject>Educational evaluation</subject><subject>Educational Quality</subject><subject>Evaluation</subject><subject>Faculty Development</subject><subject>Higher Education</subject><subject>Organizational culture</subject><subject>Quality of education</subject><subject>Statistical inference</subject><subject>Statistics</subject><subject>Survey</subject><subject>Teacher Effectiveness</subject><subject>Teacher Evaluation</subject><subject>Teaching</subject><subject>Universities</subject><issn>0951-5224</issn><issn>1468-2273</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>24P</sourceid><sourceid>WIN</sourceid><sourceid>7TQ</sourceid><recordid>eNp9kM1LAzEQxYMoWKsX70LAiwhb87XZjTep1SoFKbTnMGZTTdnutsluS_97U1c8eHAOM4f3m8fjIXRJyYDGufu0m3ZAGZf0CPWokHnCWMaPUY-olCYpY-IUnYWwJIRQykQPTR-hAbz2YBpnbMCuwpsWStfssd1C2ULj6gpDVWAIwYawslVzj2e7GreV21ofXOPiGzQY8EcJlbHn6GQBZbAXP7eP5k-j2XCcTN6eX4YPk8RwxWiSwwKEjCvLTSYLAUCJKdKikPw9MyZTKeGWFyktVE4AMsgWDMCkuVBcMGJ5H910vmtfb1obGr1ywdgyhrB1GzTLciYlE0JE9PoPuqxbX8V0kZJSMc5UHqnbjjK-DsHbhV57twK_15ToQ7v60K7-bjfCVx1svTO_4OiVcsmkSqNOO33nSrv_x0mPR9N55_kFn_uGIw</recordid><startdate>202301</startdate><enddate>202301</enddate><creator>Raffaghelli, Juliana E.</creator><creator>Grion, Valentina</creator><creator>Rossi, Marina</creator><general>Wiley</general><general>Blackwell Publishing Ltd</general><scope>24P</scope><scope>WIN</scope><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7TQ</scope><scope>DHY</scope><scope>DON</scope><orcidid>https://orcid.org/0000-0002-8753-6478</orcidid><orcidid>https://orcid.org/0000-0002-2051-1313</orcidid><orcidid>https://orcid.org/0000-0002-5115-8196</orcidid></search><sort><creationdate>202301</creationdate><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><author>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Colleges &amp; universities</topic><topic>Corporate culture</topic><topic>Data analysis</topic><topic>data cultures</topic><topic>data practices</topic><topic>Data quality</topic><topic>Data Use</topic><topic>Debates</topic><topic>Educational Assessment</topic><topic>Educational evaluation</topic><topic>Educational Quality</topic><topic>Evaluation</topic><topic>Faculty Development</topic><topic>Higher Education</topic><topic>Organizational culture</topic><topic>Quality of education</topic><topic>Statistical inference</topic><topic>Statistics</topic><topic>Survey</topic><topic>Teacher Effectiveness</topic><topic>Teacher Evaluation</topic><topic>Teaching</topic><topic>Universities</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Raffaghelli, Juliana E.</creatorcontrib><creatorcontrib>Grion, Valentina</creatorcontrib><creatorcontrib>Rossi, Marina</creatorcontrib><collection>Wiley Online Library (Open Access Collection)</collection><collection>Wiley Online Library (Open Access Collection)</collection><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><collection>PAIS Index</collection><collection>PAIS International</collection><collection>PAIS International (Ovid)</collection><jtitle>Higher education quarterly</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Raffaghelli, Juliana E.</au><au>Grion, Valentina</au><au>Rossi, Marina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1362695</ericid><atitle>Data practices in quality evaluation and assessment: Two universities at a glance</atitle><jtitle>Higher education quarterly</jtitle><date>2023-01</date><risdate>2023</risdate><volume>77</volume><issue>1</issue><spage>7</spage><epage>26</epage><pages>7-26</pages><issn>0951-5224</issn><eissn>1468-2273</eissn><abstract>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture. A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</abstract><cop>Oxford</cop><pub>Wiley</pub><doi>10.1111/hequ.12361</doi><tpages>20</tpages><orcidid>https://orcid.org/0000-0002-8753-6478</orcidid><orcidid>https://orcid.org/0000-0002-2051-1313</orcidid><orcidid>https://orcid.org/0000-0002-5115-8196</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0951-5224
ispartof Higher education quarterly, 2023-01, Vol.77 (1), p.7-26
issn 0951-5224
1468-2273
language eng
recordid cdi_proquest_miscellaneous_2782662444
source PAIS Index; Access via Wiley Online Library; EBSCOhost Education Source
subjects Colleges & universities
Corporate culture
Data analysis
data cultures
data practices
Data quality
Data Use
Debates
Educational Assessment
Educational evaluation
Educational Quality
Evaluation
Faculty Development
Higher Education
Organizational culture
Quality of education
Statistical inference
Statistics
Survey
Teacher Effectiveness
Teacher Evaluation
Teaching
Universities
title Data practices in quality evaluation and assessment: Two universities at a glance
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-26T19%3A10%3A32IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Data%20practices%20in%20quality%20evaluation%20and%20assessment:%20Two%20universities%20at%20a%20glance&rft.jtitle=Higher%20education%20quarterly&rft.au=Raffaghelli,%20Juliana%20E.&rft.date=2023-01&rft.volume=77&rft.issue=1&rft.spage=7&rft.epage=26&rft.pages=7-26&rft.issn=0951-5224&rft.eissn=1468-2273&rft_id=info:doi/10.1111/hequ.12361&rft_dat=%3Cproquest_cross%3E2766923298%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2766923298&rft_id=info:pmid/&rft_ericid=EJ1362695&rfr_iscdi=true