A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment
Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies. Scores from 21...
Gespeichert in:
Veröffentlicht in: | Academic Medicine 2008-10, Vol.83 (10 Suppl), p.S72-S75 |
---|---|
Hauptverfasser: | , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | S75 |
---|---|
container_issue | 10 Suppl |
container_start_page | S72 |
container_title | Academic Medicine |
container_volume | 83 |
creator | Kahraman, Nilufer Clauser, Brian E Margolis, Melissa J |
description | Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies.
Scores from 21,140 examinees who took the United States Medical Licensing Examination Step 2 between May 2006 and February 2007 were subjected to five weighting models: (1) a regression weights model, (2) a factor loading weights model, (3) a standardized response model, (4) an equal weights model, and (5) the operational expert-judgment weights model.
Alternative weighting procedures may have a significant impact on the reliability and validity of checklist scores.
The results suggest that the current weighting procedure is useful, and the regression-based model holds promise for practical application. The regression-based model produces scores that are more reliable than those produced by the current procedure and more strongly related to the external criteria. |
doi_str_mv | 10.1097/ACM.0b013e318183e1ac |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_69600934</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>69600934</sourcerecordid><originalsourceid>FETCH-LOGICAL-c351t-52a2a1c840abe613d57789c6ede7ae669d4f5ec766dfdf9ca20560bdff11263f3</originalsourceid><addsrcrecordid>eNpdkE1PGzEQhq2qqIS0_6BCPvW2Ybze9XqPUUQBCcSlSNxWE-84cdmP4HFAPfafs4FIlXqa9_C8jzSvEN8VLBTU1cVydbeANShNWlllNSl0n8RM1dpmFuzj5ylDAVleFOZUnDH_BgBTlfqLOFXW5lCCmYm_S-nGfocx8DjI0UvsEsUBU3ghGRL18pXCZpvCsJGcIibaBGI5sWlLssWEcoNTjAfgYBoHGtK7SLouDMFhJ_kpdB3LHUU_xh4HRxKZibmf2K_ixGPH9O145-Lh5-Wv1XV2e391s1reZk6XKmVljjkqZwvANRml27KqbO0MtVQhGVO3hS_JVca0vvW1w-lBA-vWe6Vyo72eix8f3l0cn_fEqekDO-o6HGjcc2NqA1DrYgKLD9DFkTmSb3Yx9Bj_NAqaw_TNNH3z__RT7fzo3697av-VjlvrN7nLhIg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>69600934</pqid></control><display><type>article</type><title>A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment</title><source>MEDLINE</source><source>Journals@Ovid LWW Legacy Archive</source><source>Journals@Ovid Complete</source><source>Alma/SFX Local Collection</source><creator>Kahraman, Nilufer ; Clauser, Brian E ; Margolis, Melissa J</creator><creatorcontrib>Kahraman, Nilufer ; Clauser, Brian E ; Margolis, Melissa J</creatorcontrib><description>Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies.
Scores from 21,140 examinees who took the United States Medical Licensing Examination Step 2 between May 2006 and February 2007 were subjected to five weighting models: (1) a regression weights model, (2) a factor loading weights model, (3) a standardized response model, (4) an equal weights model, and (5) the operational expert-judgment weights model.
Alternative weighting procedures may have a significant impact on the reliability and validity of checklist scores.
The results suggest that the current weighting procedure is useful, and the regression-based model holds promise for practical application. The regression-based model produces scores that are more reliable than those produced by the current procedure and more strongly related to the external criteria.</description><identifier>ISSN: 1040-2446</identifier><identifier>EISSN: 1938-808X</identifier><identifier>DOI: 10.1097/ACM.0b013e318183e1ac</identifier><identifier>PMID: 18820506</identifier><language>eng</language><publisher>United States</publisher><subject>Algorithms ; Clinical Competence - statistics & numerical data ; Cohort Studies ; Factor Analysis, Statistical ; Humans ; Judgment ; Licensure, Medical ; Models, Statistical ; Psychometrics ; Reproducibility of Results ; Retrospective Studies ; United States</subject><ispartof>Academic Medicine, 2008-10, Vol.83 (10 Suppl), p.S72-S75</ispartof><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c351t-52a2a1c840abe613d57789c6ede7ae669d4f5ec766dfdf9ca20560bdff11263f3</citedby><cites>FETCH-LOGICAL-c351t-52a2a1c840abe613d57789c6ede7ae669d4f5ec766dfdf9ca20560bdff11263f3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/18820506$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Kahraman, Nilufer</creatorcontrib><creatorcontrib>Clauser, Brian E</creatorcontrib><creatorcontrib>Margolis, Melissa J</creatorcontrib><title>A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment</title><title>Academic Medicine</title><addtitle>Acad Med</addtitle><description>Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies.
Scores from 21,140 examinees who took the United States Medical Licensing Examination Step 2 between May 2006 and February 2007 were subjected to five weighting models: (1) a regression weights model, (2) a factor loading weights model, (3) a standardized response model, (4) an equal weights model, and (5) the operational expert-judgment weights model.
Alternative weighting procedures may have a significant impact on the reliability and validity of checklist scores.
The results suggest that the current weighting procedure is useful, and the regression-based model holds promise for practical application. The regression-based model produces scores that are more reliable than those produced by the current procedure and more strongly related to the external criteria.</description><subject>Algorithms</subject><subject>Clinical Competence - statistics & numerical data</subject><subject>Cohort Studies</subject><subject>Factor Analysis, Statistical</subject><subject>Humans</subject><subject>Judgment</subject><subject>Licensure, Medical</subject><subject>Models, Statistical</subject><subject>Psychometrics</subject><subject>Reproducibility of Results</subject><subject>Retrospective Studies</subject><subject>United States</subject><issn>1040-2446</issn><issn>1938-808X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2008</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNpdkE1PGzEQhq2qqIS0_6BCPvW2Ybze9XqPUUQBCcSlSNxWE-84cdmP4HFAPfafs4FIlXqa9_C8jzSvEN8VLBTU1cVydbeANShNWlllNSl0n8RM1dpmFuzj5ylDAVleFOZUnDH_BgBTlfqLOFXW5lCCmYm_S-nGfocx8DjI0UvsEsUBU3ghGRL18pXCZpvCsJGcIibaBGI5sWlLssWEcoNTjAfgYBoHGtK7SLouDMFhJ_kpdB3LHUU_xh4HRxKZibmf2K_ixGPH9O145-Lh5-Wv1XV2e391s1reZk6XKmVljjkqZwvANRml27KqbO0MtVQhGVO3hS_JVca0vvW1w-lBA-vWe6Vyo72eix8f3l0cn_fEqekDO-o6HGjcc2NqA1DrYgKLD9DFkTmSb3Yx9Bj_NAqaw_TNNH3z__RT7fzo3697av-VjlvrN7nLhIg</recordid><startdate>20081001</startdate><enddate>20081001</enddate><creator>Kahraman, Nilufer</creator><creator>Clauser, Brian E</creator><creator>Margolis, Melissa J</creator><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope></search><sort><creationdate>20081001</creationdate><title>A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment</title><author>Kahraman, Nilufer ; Clauser, Brian E ; Margolis, Melissa J</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c351t-52a2a1c840abe613d57789c6ede7ae669d4f5ec766dfdf9ca20560bdff11263f3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2008</creationdate><topic>Algorithms</topic><topic>Clinical Competence - statistics & numerical data</topic><topic>Cohort Studies</topic><topic>Factor Analysis, Statistical</topic><topic>Humans</topic><topic>Judgment</topic><topic>Licensure, Medical</topic><topic>Models, Statistical</topic><topic>Psychometrics</topic><topic>Reproducibility of Results</topic><topic>Retrospective Studies</topic><topic>United States</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kahraman, Nilufer</creatorcontrib><creatorcontrib>Clauser, Brian E</creatorcontrib><creatorcontrib>Margolis, Melissa J</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Academic Medicine</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kahraman, Nilufer</au><au>Clauser, Brian E</au><au>Margolis, Melissa J</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment</atitle><jtitle>Academic Medicine</jtitle><addtitle>Acad Med</addtitle><date>2008-10-01</date><risdate>2008</risdate><volume>83</volume><issue>10 Suppl</issue><spage>S72</spage><epage>S75</epage><pages>S72-S75</pages><issn>1040-2446</issn><eissn>1938-808X</eissn><abstract>Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies.
Scores from 21,140 examinees who took the United States Medical Licensing Examination Step 2 between May 2006 and February 2007 were subjected to five weighting models: (1) a regression weights model, (2) a factor loading weights model, (3) a standardized response model, (4) an equal weights model, and (5) the operational expert-judgment weights model.
Alternative weighting procedures may have a significant impact on the reliability and validity of checklist scores.
The results suggest that the current weighting procedure is useful, and the regression-based model holds promise for practical application. The regression-based model produces scores that are more reliable than those produced by the current procedure and more strongly related to the external criteria.</abstract><cop>United States</cop><pmid>18820506</pmid><doi>10.1097/ACM.0b013e318183e1ac</doi><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1040-2446 |
ispartof | Academic Medicine, 2008-10, Vol.83 (10 Suppl), p.S72-S75 |
issn | 1040-2446 1938-808X |
language | eng |
recordid | cdi_proquest_miscellaneous_69600934 |
source | MEDLINE; Journals@Ovid LWW Legacy Archive; Journals@Ovid Complete; Alma/SFX Local Collection |
subjects | Algorithms Clinical Competence - statistics & numerical data Cohort Studies Factor Analysis, Statistical Humans Judgment Licensure, Medical Models, Statistical Psychometrics Reproducibility of Results Retrospective Studies United States |
title | A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T14%3A23%3A33IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20comparison%20of%20alternative%20item%20weighting%20strategies%20on%20the%20data%20gathering%20component%20of%20a%20clinical%20skills%20performance%20assessment&rft.jtitle=Academic%20Medicine&rft.au=Kahraman,%20Nilufer&rft.date=2008-10-01&rft.volume=83&rft.issue=10%20Suppl&rft.spage=S72&rft.epage=S75&rft.pages=S72-S75&rft.issn=1040-2446&rft.eissn=1938-808X&rft_id=info:doi/10.1097/ACM.0b013e318183e1ac&rft_dat=%3Cproquest_cross%3E69600934%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=69600934&rft_id=info:pmid/18820506&rfr_iscdi=true |