Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting
In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, ma...
Gespeichert in:
| Veröffentlicht in: | Educational measurement, issues and practice issues and practice, 2020, Vol.39 (1), p.45-51 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 51 |
|---|---|
| container_issue | 1 |
| container_start_page | 45 |
| container_title | Educational measurement, issues and practice |
| container_volume | 39 |
| creator | Skaggs, Gary Hein, Serge F. Wilkins, Jesse L. M. |
| description | In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard‐setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test‐centered methods that can account for the variation in borderline profiles. |
| doi_str_mv | 10.1111/emip.12228 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2376079406</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1246787</ericid><sourcerecordid>2376079406</sourcerecordid><originalsourceid>FETCH-LOGICAL-c2388-e988555ad3f693c9377c87be41a8229b13dbac3b3d963693a5481d6080a11f1f3</originalsourceid><addsrcrecordid>eNp9kE1LAzEURYMoWKsb90LAnTA1HzOTZKlt1UrFgdp1yGQyJaVNajJF-u9NHXHp2zx493AfHACuMRrhNPdma3cjTAjhJ2CAWV5kVAhyCgaIUZwdD-fgIsY1QrgoBRuAahmtW8GJVSvnY2c1rIJv7cZE2Hk4MVEHWxv46ENjwsY6AysTWh-2ymkDrYOLTrlGhQYuTNelqktw1qpNNFe_ewiWT9OP8Us2f3-ejR_mmSaU88wIzouiUA1tS0G1oIxpzmqTY8UJETWmTa00rWkjSpoIVeQcNyXiSGHc4pYOwW3fuwv-c29iJ9d-H1x6KQllJWIiR2Wi7npKBx9jMK3cBbtV4SAxkkdj8mhM_hhL8E0Pm2D1Hzh9xSQvGWcpx33-lfwc_mmS07dZ1Xd-A_jidpU</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2376079406</pqid></control><display><type>article</type><title>Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting</title><source>Access via Wiley Online Library</source><source>EBSCOhost Education Source</source><creator>Skaggs, Gary ; Hein, Serge F. ; Wilkins, Jesse L. M.</creator><creatorcontrib>Skaggs, Gary ; Hein, Serge F. ; Wilkins, Jesse L. M.</creatorcontrib><description>In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard‐setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test‐centered methods that can account for the variation in borderline profiles.</description><identifier>ISSN: 0731-1745</identifier><identifier>EISSN: 1745-3992</identifier><identifier>DOI: 10.1111/emip.12228</identifier><language>eng</language><publisher>Washington: Wiley-Blackwell</publisher><subject>borderline examinees ; Cutting Scores ; diagnostic profiles ; Diagnostic Tests ; Educational tests & measurements ; Profiles ; Standard Setting (Scoring) ; Test Items ; Testing Problems ; test‐centered standard setting</subject><ispartof>Educational measurement, issues and practice, 2020, Vol.39 (1), p.45-51</ispartof><rights>2018 by the National Council on Measurement in Education</rights><rights>2020 by the National Council on Measurement in Education</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c2388-e988555ad3f693c9377c87be41a8229b13dbac3b3d963693a5481d6080a11f1f3</citedby><cites>FETCH-LOGICAL-c2388-e988555ad3f693c9377c87be41a8229b13dbac3b3d963693a5481d6080a11f1f3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Femip.12228$$EPDF$$P50$$Gwiley$$H</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Femip.12228$$EHTML$$P50$$Gwiley$$H</linktohtml><link.rule.ids>314,780,784,1417,27924,27925,45574,45575</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1246787$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Skaggs, Gary</creatorcontrib><creatorcontrib>Hein, Serge F.</creatorcontrib><creatorcontrib>Wilkins, Jesse L. M.</creatorcontrib><title>Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting</title><title>Educational measurement, issues and practice</title><description>In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard‐setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test‐centered methods that can account for the variation in borderline profiles.</description><subject>borderline examinees</subject><subject>Cutting Scores</subject><subject>diagnostic profiles</subject><subject>Diagnostic Tests</subject><subject>Educational tests & measurements</subject><subject>Profiles</subject><subject>Standard Setting (Scoring)</subject><subject>Test Items</subject><subject>Testing Problems</subject><subject>test‐centered standard setting</subject><issn>0731-1745</issn><issn>1745-3992</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><recordid>eNp9kE1LAzEURYMoWKsb90LAnTA1HzOTZKlt1UrFgdp1yGQyJaVNajJF-u9NHXHp2zx493AfHACuMRrhNPdma3cjTAjhJ2CAWV5kVAhyCgaIUZwdD-fgIsY1QrgoBRuAahmtW8GJVSvnY2c1rIJv7cZE2Hk4MVEHWxv46ENjwsY6AysTWh-2ymkDrYOLTrlGhQYuTNelqktw1qpNNFe_ewiWT9OP8Us2f3-ejR_mmSaU88wIzouiUA1tS0G1oIxpzmqTY8UJETWmTa00rWkjSpoIVeQcNyXiSGHc4pYOwW3fuwv-c29iJ9d-H1x6KQllJWIiR2Wi7npKBx9jMK3cBbtV4SAxkkdj8mhM_hhL8E0Pm2D1Hzh9xSQvGWcpx33-lfwc_mmS07dZ1Xd-A_jidpU</recordid><startdate>2020</startdate><enddate>2020</enddate><creator>Skaggs, Gary</creator><creator>Hein, Serge F.</creator><creator>Wilkins, Jesse L. M.</creator><general>Wiley-Blackwell</general><general>Wiley Subscription Services, Inc</general><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>2020</creationdate><title>Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting</title><author>Skaggs, Gary ; Hein, Serge F. ; Wilkins, Jesse L. M.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c2388-e988555ad3f693c9377c87be41a8229b13dbac3b3d963693a5481d6080a11f1f3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>borderline examinees</topic><topic>Cutting Scores</topic><topic>diagnostic profiles</topic><topic>Diagnostic Tests</topic><topic>Educational tests & measurements</topic><topic>Profiles</topic><topic>Standard Setting (Scoring)</topic><topic>Test Items</topic><topic>Testing Problems</topic><topic>test‐centered standard setting</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Skaggs, Gary</creatorcontrib><creatorcontrib>Hein, Serge F.</creatorcontrib><creatorcontrib>Wilkins, Jesse L. M.</creatorcontrib><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><jtitle>Educational measurement, issues and practice</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Skaggs, Gary</au><au>Hein, Serge F.</au><au>Wilkins, Jesse L. M.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1246787</ericid><atitle>Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting</atitle><jtitle>Educational measurement, issues and practice</jtitle><date>2020</date><risdate>2020</risdate><volume>39</volume><issue>1</issue><spage>45</spage><epage>51</epage><pages>45-51</pages><issn>0731-1745</issn><eissn>1745-3992</eissn><abstract>In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard‐setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test‐centered methods that can account for the variation in borderline profiles.</abstract><cop>Washington</cop><pub>Wiley-Blackwell</pub><doi>10.1111/emip.12228</doi><tpages>7</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0731-1745 |
| ispartof | Educational measurement, issues and practice, 2020, Vol.39 (1), p.45-51 |
| issn | 0731-1745 1745-3992 |
| language | eng |
| recordid | cdi_proquest_journals_2376079406 |
| source | Access via Wiley Online Library; EBSCOhost Education Source |
| subjects | borderline examinees Cutting Scores diagnostic profiles Diagnostic Tests Educational tests & measurements Profiles Standard Setting (Scoring) Test Items Testing Problems test‐centered standard setting |
| title | Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T02%3A59%3A42IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Using%20Diagnostic%20Profiles%20to%20Describe%20Borderline%20Performance%20in%20Standard%20Setting&rft.jtitle=Educational%20measurement,%20issues%20and%20practice&rft.au=Skaggs,%20Gary&rft.date=2020&rft.volume=39&rft.issue=1&rft.spage=45&rft.epage=51&rft.pages=45-51&rft.issn=0731-1745&rft.eissn=1745-3992&rft_id=info:doi/10.1111/emip.12228&rft_dat=%3Cproquest_cross%3E2376079406%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2376079406&rft_id=info:pmid/&rft_ericid=EJ1246787&rfr_iscdi=true |