The cognitive process of test takers when using the script concordance test rating scale
Context Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert‐...
Gespeichert in:
| Veröffentlicht in: | Medical education 2020-04, Vol.54 (4), p.337-347 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 347 |
|---|---|
| container_issue | 4 |
| container_start_page | 337 |
| container_title | Medical education |
| container_volume | 54 |
| creator | Gawad, Nada Wood, Timothy J. Cowley, Lindsay Raiche, Isabelle |
| description | Context
Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert‐type scale. Most criticisms of the SCT relate to its rating scale but are largely theoretical. The cognitive process of test takers when selecting their responses using the SCT rating scale remains understudied, but is essential to gathering validity evidence for use of the SCT in CDM assessment.
Methods
Cases from an SCT used in a national validation study were administered to 29 residents and 14 staff surgeons. Semi‐structured cognitive interviews were then conducted with 10 residents and five staff surgeons based on the SCT results. Cognitive interview data were independently coded by two data analysts, who specifically sought to elucidate how participants mapped their internally generated responses to any of the rating scale options.
Results
Five major issues were identified with the response matching cognitive process: (a) the meaning of the ‘0' response option; (b) which response corresponds to agreement with the planned management; (c) the rationale for picking ‘±1' versus ‘±2'; (d) which response indicates the desire to undertake the planned management plus an additional procedure, and (e) the influence of time on response selection.
Conclusions
Studying how test takers (experts and trainees) interpret the SCT rating scale has revealed several issues related to inconsistent and unintended use. Revising the scale to address the variety of interpretations could help to improve the response process validity of the SCT and therefore improve the SCT's ability to be used in CDM skills assessments.
The authors studied how test‐takers (experts and trainees) understand and interpret the SCT rating scale and demonstrated several issues related to inconsistent and unintended use. |
| doi_str_mv | 10.1111/medu.14056 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2334700355</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2384093446</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3576-28ab2df8f61a5b1f21ba5b478d2cb45aa537f65f48535793fe5a5e028dffda973</originalsourceid><addsrcrecordid>eNp90E1LwzAYB_AgipvTix9ACl5EqOa1TY8y5wtMvGzgLaTpk1nt2pm0jn17Mzs9eDCXB8Lv-fPwR-iU4CsS3vUSiu6KcCySPTQkLBExzSjbR0PMsIwxIXiAjrx_wxingstDNGAkI1QkdIheZq8QmWZRl235CdHKNQa8jxobteDbqNXv4Hy0foU66nxZL6I2eG9cuWrDWm0aV-jaQK-dbrfEG13BMTqwuvJwspsjNL-bzMYP8fT5_nF8M40NE2kSU6lzWlhpE6JFTiwleZg8lQU1ORdaC5baRFguRfAZsyC0AExlYW2hs5SN0EWfG07_6MIVall6A1Wla2g6ryhjPMWYCRHo-R_61nSuDtcFJTnOGOdJUJe9Mq7x3oFVK1cutdsogtW2b7XtW333HfDZLrLLw_cv_Sk4ANKDdVnB5p8o9TS5nfehX6TBisQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2384093446</pqid></control><display><type>article</type><title>The cognitive process of test takers when using the script concordance test rating scale</title><source>MEDLINE</source><source>Wiley Online Library Journals</source><source>EBSCOhost Education Source</source><creator>Gawad, Nada ; Wood, Timothy J. ; Cowley, Lindsay ; Raiche, Isabelle</creator><creatorcontrib>Gawad, Nada ; Wood, Timothy J. ; Cowley, Lindsay ; Raiche, Isabelle</creatorcontrib><description>Context
Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert‐type scale. Most criticisms of the SCT relate to its rating scale but are largely theoretical. The cognitive process of test takers when selecting their responses using the SCT rating scale remains understudied, but is essential to gathering validity evidence for use of the SCT in CDM assessment.
Methods
Cases from an SCT used in a national validation study were administered to 29 residents and 14 staff surgeons. Semi‐structured cognitive interviews were then conducted with 10 residents and five staff surgeons based on the SCT results. Cognitive interview data were independently coded by two data analysts, who specifically sought to elucidate how participants mapped their internally generated responses to any of the rating scale options.
Results
Five major issues were identified with the response matching cognitive process: (a) the meaning of the ‘0' response option; (b) which response corresponds to agreement with the planned management; (c) the rationale for picking ‘±1' versus ‘±2'; (d) which response indicates the desire to undertake the planned management plus an additional procedure, and (e) the influence of time on response selection.
Conclusions
Studying how test takers (experts and trainees) interpret the SCT rating scale has revealed several issues related to inconsistent and unintended use. Revising the scale to address the variety of interpretations could help to improve the response process validity of the SCT and therefore improve the SCT's ability to be used in CDM skills assessments.
The authors studied how test‐takers (experts and trainees) understand and interpret the SCT rating scale and demonstrated several issues related to inconsistent and unintended use.</description><identifier>ISSN: 0308-0110</identifier><identifier>EISSN: 1365-2923</identifier><identifier>DOI: 10.1111/medu.14056</identifier><identifier>PMID: 31912562</identifier><language>eng</language><publisher>England: Wiley Subscription Services, Inc</publisher><subject>Adult ; Clinical Competence ; Clinical Decision-Making ; Cognition ; Cognition & reasoning ; Education, Medical, Graduate ; Educational Measurement ; Female ; Humans ; Interviews as Topic ; Male ; Medical education ; Prospective Studies ; Qualitative Research ; Test Taking Skills</subject><ispartof>Medical education, 2020-04, Vol.54 (4), p.337-347</ispartof><rights>2020 John Wiley & Sons Ltd and The Association for the Study of Medical Education</rights><rights>2020 John Wiley & Sons Ltd and The Association for the Study of Medical Education.</rights><rights>Copyright © 2020 John Wiley & Sons Ltd and The Association for the Study of Medical Education</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3576-28ab2df8f61a5b1f21ba5b478d2cb45aa537f65f48535793fe5a5e028dffda973</citedby><cites>FETCH-LOGICAL-c3576-28ab2df8f61a5b1f21ba5b478d2cb45aa537f65f48535793fe5a5e028dffda973</cites><orcidid>0000-0001-9177-704X ; 0000-0001-9621-3905 ; 0000-0002-3519-0153</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Fmedu.14056$$EPDF$$P50$$Gwiley$$H</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Fmedu.14056$$EHTML$$P50$$Gwiley$$H</linktohtml><link.rule.ids>314,780,784,1417,27924,27925,45574,45575</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31912562$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Gawad, Nada</creatorcontrib><creatorcontrib>Wood, Timothy J.</creatorcontrib><creatorcontrib>Cowley, Lindsay</creatorcontrib><creatorcontrib>Raiche, Isabelle</creatorcontrib><title>The cognitive process of test takers when using the script concordance test rating scale</title><title>Medical education</title><addtitle>Med Educ</addtitle><description>Context
Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert‐type scale. Most criticisms of the SCT relate to its rating scale but are largely theoretical. The cognitive process of test takers when selecting their responses using the SCT rating scale remains understudied, but is essential to gathering validity evidence for use of the SCT in CDM assessment.
Methods
Cases from an SCT used in a national validation study were administered to 29 residents and 14 staff surgeons. Semi‐structured cognitive interviews were then conducted with 10 residents and five staff surgeons based on the SCT results. Cognitive interview data were independently coded by two data analysts, who specifically sought to elucidate how participants mapped their internally generated responses to any of the rating scale options.
Results
Five major issues were identified with the response matching cognitive process: (a) the meaning of the ‘0' response option; (b) which response corresponds to agreement with the planned management; (c) the rationale for picking ‘±1' versus ‘±2'; (d) which response indicates the desire to undertake the planned management plus an additional procedure, and (e) the influence of time on response selection.
Conclusions
Studying how test takers (experts and trainees) interpret the SCT rating scale has revealed several issues related to inconsistent and unintended use. Revising the scale to address the variety of interpretations could help to improve the response process validity of the SCT and therefore improve the SCT's ability to be used in CDM skills assessments.
The authors studied how test‐takers (experts and trainees) understand and interpret the SCT rating scale and demonstrated several issues related to inconsistent and unintended use.</description><subject>Adult</subject><subject>Clinical Competence</subject><subject>Clinical Decision-Making</subject><subject>Cognition</subject><subject>Cognition & reasoning</subject><subject>Education, Medical, Graduate</subject><subject>Educational Measurement</subject><subject>Female</subject><subject>Humans</subject><subject>Interviews as Topic</subject><subject>Male</subject><subject>Medical education</subject><subject>Prospective Studies</subject><subject>Qualitative Research</subject><subject>Test Taking Skills</subject><issn>0308-0110</issn><issn>1365-2923</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp90E1LwzAYB_AgipvTix9ACl5EqOa1TY8y5wtMvGzgLaTpk1nt2pm0jn17Mzs9eDCXB8Lv-fPwR-iU4CsS3vUSiu6KcCySPTQkLBExzSjbR0PMsIwxIXiAjrx_wxingstDNGAkI1QkdIheZq8QmWZRl235CdHKNQa8jxobteDbqNXv4Hy0foU66nxZL6I2eG9cuWrDWm0aV-jaQK-dbrfEG13BMTqwuvJwspsjNL-bzMYP8fT5_nF8M40NE2kSU6lzWlhpE6JFTiwleZg8lQU1ORdaC5baRFguRfAZsyC0AExlYW2hs5SN0EWfG07_6MIVall6A1Wla2g6ryhjPMWYCRHo-R_61nSuDtcFJTnOGOdJUJe9Mq7x3oFVK1cutdsogtW2b7XtW333HfDZLrLLw_cv_Sk4ANKDdVnB5p8o9TS5nfehX6TBisQ</recordid><startdate>202004</startdate><enddate>202004</enddate><creator>Gawad, Nada</creator><creator>Wood, Timothy J.</creator><creator>Cowley, Lindsay</creator><creator>Raiche, Isabelle</creator><general>Wiley Subscription Services, Inc</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>K9.</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0001-9177-704X</orcidid><orcidid>https://orcid.org/0000-0001-9621-3905</orcidid><orcidid>https://orcid.org/0000-0002-3519-0153</orcidid></search><sort><creationdate>202004</creationdate><title>The cognitive process of test takers when using the script concordance test rating scale</title><author>Gawad, Nada ; Wood, Timothy J. ; Cowley, Lindsay ; Raiche, Isabelle</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3576-28ab2df8f61a5b1f21ba5b478d2cb45aa537f65f48535793fe5a5e028dffda973</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Adult</topic><topic>Clinical Competence</topic><topic>Clinical Decision-Making</topic><topic>Cognition</topic><topic>Cognition & reasoning</topic><topic>Education, Medical, Graduate</topic><topic>Educational Measurement</topic><topic>Female</topic><topic>Humans</topic><topic>Interviews as Topic</topic><topic>Male</topic><topic>Medical education</topic><topic>Prospective Studies</topic><topic>Qualitative Research</topic><topic>Test Taking Skills</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Gawad, Nada</creatorcontrib><creatorcontrib>Wood, Timothy J.</creatorcontrib><creatorcontrib>Cowley, Lindsay</creatorcontrib><creatorcontrib>Raiche, Isabelle</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>MEDLINE - Academic</collection><jtitle>Medical education</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Gawad, Nada</au><au>Wood, Timothy J.</au><au>Cowley, Lindsay</au><au>Raiche, Isabelle</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The cognitive process of test takers when using the script concordance test rating scale</atitle><jtitle>Medical education</jtitle><addtitle>Med Educ</addtitle><date>2020-04</date><risdate>2020</risdate><volume>54</volume><issue>4</issue><spage>337</spage><epage>347</epage><pages>337-347</pages><issn>0308-0110</issn><eissn>1365-2923</eissn><abstract>Context
Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert‐type scale. Most criticisms of the SCT relate to its rating scale but are largely theoretical. The cognitive process of test takers when selecting their responses using the SCT rating scale remains understudied, but is essential to gathering validity evidence for use of the SCT in CDM assessment.
Methods
Cases from an SCT used in a national validation study were administered to 29 residents and 14 staff surgeons. Semi‐structured cognitive interviews were then conducted with 10 residents and five staff surgeons based on the SCT results. Cognitive interview data were independently coded by two data analysts, who specifically sought to elucidate how participants mapped their internally generated responses to any of the rating scale options.
Results
Five major issues were identified with the response matching cognitive process: (a) the meaning of the ‘0' response option; (b) which response corresponds to agreement with the planned management; (c) the rationale for picking ‘±1' versus ‘±2'; (d) which response indicates the desire to undertake the planned management plus an additional procedure, and (e) the influence of time on response selection.
Conclusions
Studying how test takers (experts and trainees) interpret the SCT rating scale has revealed several issues related to inconsistent and unintended use. Revising the scale to address the variety of interpretations could help to improve the response process validity of the SCT and therefore improve the SCT's ability to be used in CDM skills assessments.
The authors studied how test‐takers (experts and trainees) understand and interpret the SCT rating scale and demonstrated several issues related to inconsistent and unintended use.</abstract><cop>England</cop><pub>Wiley Subscription Services, Inc</pub><pmid>31912562</pmid><doi>10.1111/medu.14056</doi><tpages>11</tpages><orcidid>https://orcid.org/0000-0001-9177-704X</orcidid><orcidid>https://orcid.org/0000-0001-9621-3905</orcidid><orcidid>https://orcid.org/0000-0002-3519-0153</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0308-0110 |
| ispartof | Medical education, 2020-04, Vol.54 (4), p.337-347 |
| issn | 0308-0110 1365-2923 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_2334700355 |
| source | MEDLINE; Wiley Online Library Journals; EBSCOhost Education Source |
| subjects | Adult Clinical Competence Clinical Decision-Making Cognition Cognition & reasoning Education, Medical, Graduate Educational Measurement Female Humans Interviews as Topic Male Medical education Prospective Studies Qualitative Research Test Taking Skills |
| title | The cognitive process of test takers when using the script concordance test rating scale |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-01T10%3A23%3A54IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20cognitive%20process%20of%20test%20takers%20when%20using%20the%20script%20concordance%20test%20rating%20scale&rft.jtitle=Medical%20education&rft.au=Gawad,%20Nada&rft.date=2020-04&rft.volume=54&rft.issue=4&rft.spage=337&rft.epage=347&rft.pages=337-347&rft.issn=0308-0110&rft.eissn=1365-2923&rft_id=info:doi/10.1111/medu.14056&rft_dat=%3Cproquest_cross%3E2384093446%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2384093446&rft_id=info:pmid/31912562&rfr_iscdi=true |