Data practices in quality evaluation and assessment: Two universities at a glance
As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enha...
Gespeichert in:
Veröffentlicht in: | Higher education quarterly 2023-01, Vol.77 (1), p.7-26 |
---|---|
Hauptverfasser: | , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 26 |
---|---|
container_issue | 1 |
container_start_page | 7 |
container_title | Higher education quarterly |
container_volume | 77 |
creator | Raffaghelli, Juliana E. Grion, Valentina Rossi, Marina |
description | As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture.
A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instit |
doi_str_mv | 10.1111/hequ.12361 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2782662444</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1362695</ericid><sourcerecordid>2766923298</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</originalsourceid><addsrcrecordid>eNp9kM1LAzEQxYMoWKsX70LAiwhb87XZjTep1SoFKbTnMGZTTdnutsluS_97U1c8eHAOM4f3m8fjIXRJyYDGufu0m3ZAGZf0CPWokHnCWMaPUY-olCYpY-IUnYWwJIRQykQPTR-hAbz2YBpnbMCuwpsWStfssd1C2ULj6gpDVWAIwYawslVzj2e7GreV21ofXOPiGzQY8EcJlbHn6GQBZbAXP7eP5k-j2XCcTN6eX4YPk8RwxWiSwwKEjCvLTSYLAUCJKdKikPw9MyZTKeGWFyktVE4AMsgWDMCkuVBcMGJ5H910vmtfb1obGr1ywdgyhrB1GzTLciYlE0JE9PoPuqxbX8V0kZJSMc5UHqnbjjK-DsHbhV57twK_15ToQ7v60K7-bjfCVx1svTO_4OiVcsmkSqNOO33nSrv_x0mPR9N55_kFn_uGIw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2766923298</pqid></control><display><type>article</type><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><source>PAIS Index</source><source>Access via Wiley Online Library</source><source>EBSCOhost Education Source</source><creator>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</creator><creatorcontrib>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</creatorcontrib><description>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture.
A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</description><identifier>ISSN: 0951-5224</identifier><identifier>EISSN: 1468-2273</identifier><identifier>DOI: 10.1111/hequ.12361</identifier><language>eng</language><publisher>Oxford: Wiley</publisher><subject>Colleges & universities ; Corporate culture ; Data analysis ; data cultures ; data practices ; Data quality ; Data Use ; Debates ; Educational Assessment ; Educational evaluation ; Educational Quality ; Evaluation ; Faculty Development ; Higher Education ; Organizational culture ; Quality of education ; Statistical inference ; Statistics ; Survey ; Teacher Effectiveness ; Teacher Evaluation ; Teaching ; Universities</subject><ispartof>Higher education quarterly, 2023-01, Vol.77 (1), p.7-26</ispartof><rights>2021 The Authors. published by John Wiley & Sons Ltd.</rights><rights>2021. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</citedby><cites>FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</cites><orcidid>0000-0002-8753-6478 ; 0000-0002-2051-1313 ; 0000-0002-5115-8196</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Fhequ.12361$$EPDF$$P50$$Gwiley$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Fhequ.12361$$EHTML$$P50$$Gwiley$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,1417,27866,27924,27925,45574,45575</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1362695$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Raffaghelli, Juliana E.</creatorcontrib><creatorcontrib>Grion, Valentina</creatorcontrib><creatorcontrib>Rossi, Marina</creatorcontrib><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><title>Higher education quarterly</title><description>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture.
A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</description><subject>Colleges & universities</subject><subject>Corporate culture</subject><subject>Data analysis</subject><subject>data cultures</subject><subject>data practices</subject><subject>Data quality</subject><subject>Data Use</subject><subject>Debates</subject><subject>Educational Assessment</subject><subject>Educational evaluation</subject><subject>Educational Quality</subject><subject>Evaluation</subject><subject>Faculty Development</subject><subject>Higher Education</subject><subject>Organizational culture</subject><subject>Quality of education</subject><subject>Statistical inference</subject><subject>Statistics</subject><subject>Survey</subject><subject>Teacher Effectiveness</subject><subject>Teacher Evaluation</subject><subject>Teaching</subject><subject>Universities</subject><issn>0951-5224</issn><issn>1468-2273</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>24P</sourceid><sourceid>WIN</sourceid><sourceid>7TQ</sourceid><recordid>eNp9kM1LAzEQxYMoWKsX70LAiwhb87XZjTep1SoFKbTnMGZTTdnutsluS_97U1c8eHAOM4f3m8fjIXRJyYDGufu0m3ZAGZf0CPWokHnCWMaPUY-olCYpY-IUnYWwJIRQykQPTR-hAbz2YBpnbMCuwpsWStfssd1C2ULj6gpDVWAIwYawslVzj2e7GreV21ofXOPiGzQY8EcJlbHn6GQBZbAXP7eP5k-j2XCcTN6eX4YPk8RwxWiSwwKEjCvLTSYLAUCJKdKikPw9MyZTKeGWFyktVE4AMsgWDMCkuVBcMGJ5H910vmtfb1obGr1ywdgyhrB1GzTLciYlE0JE9PoPuqxbX8V0kZJSMc5UHqnbjjK-DsHbhV57twK_15ToQ7v60K7-bjfCVx1svTO_4OiVcsmkSqNOO33nSrv_x0mPR9N55_kFn_uGIw</recordid><startdate>202301</startdate><enddate>202301</enddate><creator>Raffaghelli, Juliana E.</creator><creator>Grion, Valentina</creator><creator>Rossi, Marina</creator><general>Wiley</general><general>Blackwell Publishing Ltd</general><scope>24P</scope><scope>WIN</scope><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7TQ</scope><scope>DHY</scope><scope>DON</scope><orcidid>https://orcid.org/0000-0002-8753-6478</orcidid><orcidid>https://orcid.org/0000-0002-2051-1313</orcidid><orcidid>https://orcid.org/0000-0002-5115-8196</orcidid></search><sort><creationdate>202301</creationdate><title>Data practices in quality evaluation and assessment: Two universities at a glance</title><author>Raffaghelli, Juliana E. ; Grion, Valentina ; Rossi, Marina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3921-8afa46afa78c76d4aa10cd5dd63b7cc79503e3d51d980aa7a7f2aac58493420e3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Colleges & universities</topic><topic>Corporate culture</topic><topic>Data analysis</topic><topic>data cultures</topic><topic>data practices</topic><topic>Data quality</topic><topic>Data Use</topic><topic>Debates</topic><topic>Educational Assessment</topic><topic>Educational evaluation</topic><topic>Educational Quality</topic><topic>Evaluation</topic><topic>Faculty Development</topic><topic>Higher Education</topic><topic>Organizational culture</topic><topic>Quality of education</topic><topic>Statistical inference</topic><topic>Statistics</topic><topic>Survey</topic><topic>Teacher Effectiveness</topic><topic>Teacher Evaluation</topic><topic>Teaching</topic><topic>Universities</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Raffaghelli, Juliana E.</creatorcontrib><creatorcontrib>Grion, Valentina</creatorcontrib><creatorcontrib>Rossi, Marina</creatorcontrib><collection>Wiley Online Library (Open Access Collection)</collection><collection>Wiley Online Library (Open Access Collection)</collection><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><collection>PAIS Index</collection><collection>PAIS International</collection><collection>PAIS International (Ovid)</collection><jtitle>Higher education quarterly</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Raffaghelli, Juliana E.</au><au>Grion, Valentina</au><au>Rossi, Marina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1362695</ericid><atitle>Data practices in quality evaluation and assessment: Two universities at a glance</atitle><jtitle>Higher education quarterly</jtitle><date>2023-01</date><risdate>2023</risdate><volume>77</volume><issue>1</issue><spage>7</spage><epage>26</epage><pages>7-26</pages><issn>0951-5224</issn><eissn>1468-2273</eissn><abstract>As the debate on data in the society and in education grows the attention on data‐trace as ‘primary material’ for governance, educational quality and innovation falls under the spotlights. In this context, HEIs have been put under pressure to adopt quantitative metrics and evaluation approaches enhancing the massive collection of trace data. Nonetheless, each university overall, and the academics specifically, might respond differently to this context of innovation. The present article aims to explore data practices in two higher education institutions. Two relevant areas for the imaginaries related to data and quantification were explored: (a) evaluation of quality in teaching and learning; (b) data to support assessment. The study is based on a survey distributed to the whole university teaching staff of two institutions. Descriptive and inferential statistics comparing multivariate sample means (MANOVA) were applied to 601 responses collected. The results indicated the prevalence of institutionally consolidated data practices relative to quality teaching evaluation, with fragmentation and isolation in some emerging data practices connected to decision‐making and teaching and learning. Moreover, each of the universities revealed distinct institutional profiles which could be put in connection with the organisational culture. The results are discussed in light of the potential strategies at the institutional level, particularly regarding faculty development as means to build a visible, contextualised data culture.
A medida que aumenta el debate sobre los datos en la sociedad y en la educación, la atención sobre el rastreo de datos como ‘material primario’ para la gobernanza, la calidad educativa y la innovación recibe cada vez más atención. En este contexto, se ha presionado a las IES para que adopten métricas cuantitativas y enfoques de evaluación que potencien la recopilación masiva de datos de seguimiento. Sin embargo, cada universidad en general, y los académicos en particular, pueden responder de forma diferente a este contexto de innovación. El presente artículo pretende explorar las prácticas de datos en dos instituciones de educación superior. Se exploran dos áreas relevantes para los imaginarios relacionados con los datos y la cuantificación: (a) la evaluación de la calidad en la enseñanza y el aprendizaje; (b) los datos para apoyar la evaluación. El estudio se basa en una encuesta distribuida a todo el profesorado universitario de las dos instituciones. Se aplicaron estadísticas descriptivas e inferenciales de comparación de medias muestrales multivariadas (MANOVA) a 601 respuestas recogidas. Los resultados indicaron la prevalencia de prácticas de datos consolidadas institucionalmente en relación con la evaluación de la calidad de la enseñanza, con fragmentación y aislamiento en algunas prácticas de datos emergentes relacionadas con la toma de decisiones y la enseñanza y el aprendizaje. Además, cada una de las universidades reveló perfiles institucionales distintos que podrían ponerse en relación con la cultura organizativa. Los resultados se discuten a la luz de las posibles estrategias a nivel institucional, especialmente en lo que respecta al desarrollo del profesorado como medio para construir una cultura de datos visible y contextualizada.</abstract><cop>Oxford</cop><pub>Wiley</pub><doi>10.1111/hequ.12361</doi><tpages>20</tpages><orcidid>https://orcid.org/0000-0002-8753-6478</orcidid><orcidid>https://orcid.org/0000-0002-2051-1313</orcidid><orcidid>https://orcid.org/0000-0002-5115-8196</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0951-5224 |
ispartof | Higher education quarterly, 2023-01, Vol.77 (1), p.7-26 |
issn | 0951-5224 1468-2273 |
language | eng |
recordid | cdi_proquest_miscellaneous_2782662444 |
source | PAIS Index; Access via Wiley Online Library; EBSCOhost Education Source |
subjects | Colleges & universities Corporate culture Data analysis data cultures data practices Data quality Data Use Debates Educational Assessment Educational evaluation Educational Quality Evaluation Faculty Development Higher Education Organizational culture Quality of education Statistical inference Statistics Survey Teacher Effectiveness Teacher Evaluation Teaching Universities |
title | Data practices in quality evaluation and assessment: Two universities at a glance |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-26T19%3A10%3A32IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Data%20practices%20in%20quality%20evaluation%20and%20assessment:%20Two%20universities%20at%20a%20glance&rft.jtitle=Higher%20education%20quarterly&rft.au=Raffaghelli,%20Juliana%20E.&rft.date=2023-01&rft.volume=77&rft.issue=1&rft.spage=7&rft.epage=26&rft.pages=7-26&rft.issn=0951-5224&rft.eissn=1468-2273&rft_id=info:doi/10.1111/hequ.12361&rft_dat=%3Cproquest_cross%3E2766923298%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2766923298&rft_id=info:pmid/&rft_ericid=EJ1362695&rfr_iscdi=true |