Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration
Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes....
Gespeichert in:
| Veröffentlicht in: | Journal of educational measurement 2021-12, Vol.58 (4), p.515-537 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 537 |
|---|---|
| container_issue | 4 |
| container_start_page | 515 |
| container_title | Journal of educational measurement |
| container_volume | 58 |
| creator | Yaneva, Victoria Clauser, Brian E. Morales, Amy Paniagua, Miguel |
| description | Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye‐tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye‐tracking study designed to evaluate how the presence of the options in multiple‐choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study. |
| doi_str_mv | 10.1111/jedm.12304 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2635327207</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1327572</ericid><sourcerecordid>2635327207</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3234-86b28461c237cbc915ad61c38e6870599f4e0de927461a8ce987cd944573cfae3</originalsourceid><addsrcrecordid>eNp9kMtOwzAQRS0EEqWwYY9kiR1Sih9xnLCr2vCoigCpZRu5jtO6zaPYiVB2fALfyJfgEMSS2Xiu58y1dQE4x2iEXV1vVVqMMKHIPwADzH3m0Sj0D8EAIUI8FDB2DE6s3SKEGWd4AHZLq8s1jFv19fG5MELuOjkVtYDCwmdhalhlsN4o-Cpyneq6hWOzbgpV1jCrDHxs8lrv8257sqm0VPClUbbWVWlv4BhOVeG62oju5hQcZSK36uz3HILlbbyY3Hvzp7uHyXjuSUqo74XBioR-gCWhXK5khJlInaKhCkKOWBRlvkKpigh3kAilikIu08j3GacyE4oOwWXvuzfVW_ebZFs1pnRPJiSgjBJOEHfUVU9JU1lrVJbsjS6EaROMki7MpAsz-QnTwRc9rIyWf2A8w86MceLmuJ-_61y1_zgls3j62Ht-A9mBga0</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2635327207</pqid></control><display><type>article</type><title>Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration</title><source>Applied Social Sciences Index & Abstracts (ASSIA)</source><source>Education Source</source><source>Wiley Online Library Journals Frontfile Complete</source><creator>Yaneva, Victoria ; Clauser, Brian E. ; Morales, Amy ; Paniagua, Miguel</creator><creatorcontrib>Yaneva, Victoria ; Clauser, Brian E. ; Morales, Amy ; Paniagua, Miguel</creatorcontrib><description>Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye‐tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye‐tracking study designed to evaluate how the presence of the options in multiple‐choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study.</description><identifier>ISSN: 0022-0655</identifier><identifier>EISSN: 1745-3984</identifier><identifier>DOI: 10.1111/jedm.12304</identifier><language>eng</language><publisher>Madison: Wiley</publisher><subject>Clinical decision making ; Cognition ; Cognitive Processes ; Educational evaluation ; Educational tests & measurements ; Eye Movements ; Inferences ; Medical Students ; Multiple Choice Tests ; Scores ; Student Evaluation ; Technology ; Test Construction ; Test Items ; Test scores ; Test Validity ; Thinking Skills ; Tracking ; Usefulness ; Validity</subject><ispartof>Journal of educational measurement, 2021-12, Vol.58 (4), p.515-537</ispartof><rights>2021 National Board of Medical Examiners. published by Wiley Periodicals LLC on behalf of National Council on Measurement in Education</rights><rights>2021. This article is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3234-86b28461c237cbc915ad61c38e6870599f4e0de927461a8ce987cd944573cfae3</citedby><cites>FETCH-LOGICAL-c3234-86b28461c237cbc915ad61c38e6870599f4e0de927461a8ce987cd944573cfae3</cites><orcidid>0000-0003-2307-4873</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Fjedm.12304$$EPDF$$P50$$Gwiley$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Fjedm.12304$$EHTML$$P50$$Gwiley$$Hfree_for_read</linktohtml><link.rule.ids>314,776,780,1411,27901,27902,30976,45550,45551</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1327572$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Yaneva, Victoria</creatorcontrib><creatorcontrib>Clauser, Brian E.</creatorcontrib><creatorcontrib>Morales, Amy</creatorcontrib><creatorcontrib>Paniagua, Miguel</creatorcontrib><title>Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration</title><title>Journal of educational measurement</title><description>Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye‐tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye‐tracking study designed to evaluate how the presence of the options in multiple‐choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study.</description><subject>Clinical decision making</subject><subject>Cognition</subject><subject>Cognitive Processes</subject><subject>Educational evaluation</subject><subject>Educational tests & measurements</subject><subject>Eye Movements</subject><subject>Inferences</subject><subject>Medical Students</subject><subject>Multiple Choice Tests</subject><subject>Scores</subject><subject>Student Evaluation</subject><subject>Technology</subject><subject>Test Construction</subject><subject>Test Items</subject><subject>Test scores</subject><subject>Test Validity</subject><subject>Thinking Skills</subject><subject>Tracking</subject><subject>Usefulness</subject><subject>Validity</subject><issn>0022-0655</issn><issn>1745-3984</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>24P</sourceid><sourceid>7QJ</sourceid><recordid>eNp9kMtOwzAQRS0EEqWwYY9kiR1Sih9xnLCr2vCoigCpZRu5jtO6zaPYiVB2fALfyJfgEMSS2Xiu58y1dQE4x2iEXV1vVVqMMKHIPwADzH3m0Sj0D8EAIUI8FDB2DE6s3SKEGWd4AHZLq8s1jFv19fG5MELuOjkVtYDCwmdhalhlsN4o-Cpyneq6hWOzbgpV1jCrDHxs8lrv8257sqm0VPClUbbWVWlv4BhOVeG62oju5hQcZSK36uz3HILlbbyY3Hvzp7uHyXjuSUqo74XBioR-gCWhXK5khJlInaKhCkKOWBRlvkKpigh3kAilikIu08j3GacyE4oOwWXvuzfVW_ebZFs1pnRPJiSgjBJOEHfUVU9JU1lrVJbsjS6EaROMki7MpAsz-QnTwRc9rIyWf2A8w86MceLmuJ-_61y1_zgls3j62Ht-A9mBga0</recordid><startdate>20211201</startdate><enddate>20211201</enddate><creator>Yaneva, Victoria</creator><creator>Clauser, Brian E.</creator><creator>Morales, Amy</creator><creator>Paniagua, Miguel</creator><general>Wiley</general><general>Wiley Subscription Services, Inc</general><scope>24P</scope><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QJ</scope><orcidid>https://orcid.org/0000-0003-2307-4873</orcidid></search><sort><creationdate>20211201</creationdate><title>Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration</title><author>Yaneva, Victoria ; Clauser, Brian E. ; Morales, Amy ; Paniagua, Miguel</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3234-86b28461c237cbc915ad61c38e6870599f4e0de927461a8ce987cd944573cfae3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Clinical decision making</topic><topic>Cognition</topic><topic>Cognitive Processes</topic><topic>Educational evaluation</topic><topic>Educational tests & measurements</topic><topic>Eye Movements</topic><topic>Inferences</topic><topic>Medical Students</topic><topic>Multiple Choice Tests</topic><topic>Scores</topic><topic>Student Evaluation</topic><topic>Technology</topic><topic>Test Construction</topic><topic>Test Items</topic><topic>Test scores</topic><topic>Test Validity</topic><topic>Thinking Skills</topic><topic>Tracking</topic><topic>Usefulness</topic><topic>Validity</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Yaneva, Victoria</creatorcontrib><creatorcontrib>Clauser, Brian E.</creatorcontrib><creatorcontrib>Morales, Amy</creatorcontrib><creatorcontrib>Paniagua, Miguel</creatorcontrib><collection>Wiley-Blackwell Open Access Titles</collection><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><collection>Applied Social Sciences Index & Abstracts (ASSIA)</collection><jtitle>Journal of educational measurement</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Yaneva, Victoria</au><au>Clauser, Brian E.</au><au>Morales, Amy</au><au>Paniagua, Miguel</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1327572</ericid><atitle>Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration</atitle><jtitle>Journal of educational measurement</jtitle><date>2021-12-01</date><risdate>2021</risdate><volume>58</volume><issue>4</issue><spage>515</spage><epage>537</epage><pages>515-537</pages><issn>0022-0655</issn><eissn>1745-3984</eissn><abstract>Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye‐tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye‐tracking study designed to evaluate how the presence of the options in multiple‐choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study.</abstract><cop>Madison</cop><pub>Wiley</pub><doi>10.1111/jedm.12304</doi><tpages>23</tpages><orcidid>https://orcid.org/0000-0003-2307-4873</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0022-0655 |
| ispartof | Journal of educational measurement, 2021-12, Vol.58 (4), p.515-537 |
| issn | 0022-0655 1745-3984 |
| language | eng |
| recordid | cdi_proquest_journals_2635327207 |
| source | Applied Social Sciences Index & Abstracts (ASSIA); Education Source; Wiley Online Library Journals Frontfile Complete |
| subjects | Clinical decision making Cognition Cognitive Processes Educational evaluation Educational tests & measurements Eye Movements Inferences Medical Students Multiple Choice Tests Scores Student Evaluation Technology Test Construction Test Items Test scores Test Validity Thinking Skills Tracking Usefulness Validity |
| title | Using Eye‐Tracking Data as Part of the Validity Argument for Multiple‐Choice Questions: A Demonstration |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-02T20%3A23%3A31IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Using%20Eye%E2%80%90Tracking%20Data%20as%20Part%20of%20the%20Validity%20Argument%20for%20Multiple%E2%80%90Choice%20Questions:%20A%20Demonstration&rft.jtitle=Journal%20of%20educational%20measurement&rft.au=Yaneva,%20Victoria&rft.date=2021-12-01&rft.volume=58&rft.issue=4&rft.spage=515&rft.epage=537&rft.pages=515-537&rft.issn=0022-0655&rft.eissn=1745-3984&rft_id=info:doi/10.1111/jedm.12304&rft_dat=%3Cproquest_cross%3E2635327207%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2635327207&rft_id=info:pmid/&rft_ericid=EJ1327572&rfr_iscdi=true |