The use of scoring rubrics to determine clinical performance in the operating suite
This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite. Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College o...
Gespeichert in:
| Veröffentlicht in: | Nurse education today 2009, Vol.29 (1), p.73-82 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 82 |
|---|---|
| container_issue | 1 |
| container_start_page | 73 |
| container_title | Nurse education today |
| container_volume | 29 |
| creator | Nicholson, Patricia Gillis, Shelley Dunning, A.M. Trisha |
| description | This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.
Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.
The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (
Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.
The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite. |
| doi_str_mv | 10.1016/j.nedt.2008.06.011 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_764377479</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0260691708000798</els_id><sourcerecordid>57264413</sourcerecordid><originalsourceid>FETCH-LOGICAL-c444t-c36528b9660a49319f4f7e0ce37843d92ef3e2633438c954b112cf50f78111f3</originalsourceid><addsrcrecordid>eNqFkc1KJDEURoOMaE_rC8xiCLNwVlXmr5IUuBFRRxBc2PtQnbrRNFWVNkkN-Pam6YaBWegqcDnfx809CP2gpKaEystNPUGfa0aIromsCaVHaEEbziqmWv4NLQiTpJItVafoe0obUkDF-Ak6pVo1TcPaBXpevQKeE-DgcLIh-ukFx3kdvU04B9xDhjj6CbAd_ORtN-AtRBfi2E0WsJ9wLvlQZl3eRdPsM5yhY9cNCc4P7xKt7m5XN3-qx6f7h5vrx8oKIXJluWyYXrdSkk60nLZOOAXEAlda8L5l4Dgwybng2raNWFPKrGuIU5pS6vgS_d7XbmN4myFlM_pkYRi6CcKcjJKCKyXKKZbo4lNSSs3KSuRLsFFMCkF5AX_9B27CHKfyW1N0MCKY0AVie8jGkFIEZ7bRj118N5SYnUGzMTuDu4g2RJpisIR-Hprn9Qj9v8hBWQGu9gCU0_71EE2yHoqN3kew2fTBf9b_Acxoqpc</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>200204248</pqid></control><display><type>article</type><title>The use of scoring rubrics to determine clinical performance in the operating suite</title><source>MEDLINE</source><source>Applied Social Sciences Index & Abstracts (ASSIA)</source><source>ScienceDirect Journals (5 years ago - present)</source><creator>Nicholson, Patricia ; Gillis, Shelley ; Dunning, A.M. Trisha</creator><creatorcontrib>Nicholson, Patricia ; Gillis, Shelley ; Dunning, A.M. Trisha</creatorcontrib><description>This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.
Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.
The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (
Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.
The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.</description><identifier>ISSN: 0260-6917</identifier><identifier>EISSN: 1532-2793</identifier><identifier>DOI: 10.1016/j.nedt.2008.06.011</identifier><identifier>PMID: 18755529</identifier><language>eng</language><publisher>Scotland: Elsevier Ltd</publisher><subject>Analysis of Variance ; Assessment ; Calibration ; Career Mobility ; Clinical Competence - standards ; Clinical standards ; Competence ; Competency-based assessment ; Construct Validity ; Core competencies ; Data Interpretation, Statistical ; Educational Measurement - methods ; Educational Measurement - standards ; Employee Performance Appraisal - methods ; Employee Performance Appraisal - standards ; Humans ; Inter-rater reliability ; Interrater Reliability ; Item separation index ; Models, Nursing ; Nurse's Role ; Nurses ; Nursing ; Nursing education ; Nursing Evaluation Research ; Nursing Staff, Hospital - education ; Nursing Staff, Hospital - standards ; Observer Variation ; Operating Room Nursing - education ; Operating Room Nursing - standards ; Operating theatres ; Practice Guidelines as Topic ; Psychometrics ; Psychomotor Performance ; Reliability ; Rubrics ; Scoring Rubrics ; Skills ; Validation studies ; Validity ; Victoria ; Videotape Recording - methods ; Videotape Recording - standards</subject><ispartof>Nurse education today, 2009, Vol.29 (1), p.73-82</ispartof><rights>2008 Elsevier Ltd</rights><rights>Copyright Elsevier Science Ltd. Jan 2009</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c444t-c36528b9660a49319f4f7e0ce37843d92ef3e2633438c954b112cf50f78111f3</citedby><cites>FETCH-LOGICAL-c444t-c36528b9660a49319f4f7e0ce37843d92ef3e2633438c954b112cf50f78111f3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.nedt.2008.06.011$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,780,784,3550,4024,27923,27924,27925,30999,31000,45995</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/18755529$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Nicholson, Patricia</creatorcontrib><creatorcontrib>Gillis, Shelley</creatorcontrib><creatorcontrib>Dunning, A.M. Trisha</creatorcontrib><title>The use of scoring rubrics to determine clinical performance in the operating suite</title><title>Nurse education today</title><addtitle>Nurse Educ Today</addtitle><description>This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.
Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.
The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (
Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.
The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.</description><subject>Analysis of Variance</subject><subject>Assessment</subject><subject>Calibration</subject><subject>Career Mobility</subject><subject>Clinical Competence - standards</subject><subject>Clinical standards</subject><subject>Competence</subject><subject>Competency-based assessment</subject><subject>Construct Validity</subject><subject>Core competencies</subject><subject>Data Interpretation, Statistical</subject><subject>Educational Measurement - methods</subject><subject>Educational Measurement - standards</subject><subject>Employee Performance Appraisal - methods</subject><subject>Employee Performance Appraisal - standards</subject><subject>Humans</subject><subject>Inter-rater reliability</subject><subject>Interrater Reliability</subject><subject>Item separation index</subject><subject>Models, Nursing</subject><subject>Nurse's Role</subject><subject>Nurses</subject><subject>Nursing</subject><subject>Nursing education</subject><subject>Nursing Evaluation Research</subject><subject>Nursing Staff, Hospital - education</subject><subject>Nursing Staff, Hospital - standards</subject><subject>Observer Variation</subject><subject>Operating Room Nursing - education</subject><subject>Operating Room Nursing - standards</subject><subject>Operating theatres</subject><subject>Practice Guidelines as Topic</subject><subject>Psychometrics</subject><subject>Psychomotor Performance</subject><subject>Reliability</subject><subject>Rubrics</subject><subject>Scoring Rubrics</subject><subject>Skills</subject><subject>Validation studies</subject><subject>Validity</subject><subject>Victoria</subject><subject>Videotape Recording - methods</subject><subject>Videotape Recording - standards</subject><issn>0260-6917</issn><issn>1532-2793</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2009</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>7QJ</sourceid><recordid>eNqFkc1KJDEURoOMaE_rC8xiCLNwVlXmr5IUuBFRRxBc2PtQnbrRNFWVNkkN-Pam6YaBWegqcDnfx809CP2gpKaEystNPUGfa0aIromsCaVHaEEbziqmWv4NLQiTpJItVafoe0obUkDF-Ak6pVo1TcPaBXpevQKeE-DgcLIh-ukFx3kdvU04B9xDhjj6CbAd_ORtN-AtRBfi2E0WsJ9wLvlQZl3eRdPsM5yhY9cNCc4P7xKt7m5XN3-qx6f7h5vrx8oKIXJluWyYXrdSkk60nLZOOAXEAlda8L5l4Dgwybng2raNWFPKrGuIU5pS6vgS_d7XbmN4myFlM_pkYRi6CcKcjJKCKyXKKZbo4lNSSs3KSuRLsFFMCkF5AX_9B27CHKfyW1N0MCKY0AVie8jGkFIEZ7bRj118N5SYnUGzMTuDu4g2RJpisIR-Hprn9Qj9v8hBWQGu9gCU0_71EE2yHoqN3kew2fTBf9b_Acxoqpc</recordid><startdate>2009</startdate><enddate>2009</enddate><creator>Nicholson, Patricia</creator><creator>Gillis, Shelley</creator><creator>Dunning, A.M. Trisha</creator><general>Elsevier Ltd</general><general>Elsevier Science Ltd</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QJ</scope><scope>ASE</scope><scope>FPQ</scope><scope>K6X</scope><scope>NAPCQ</scope><scope>7X8</scope></search><sort><creationdate>2009</creationdate><title>The use of scoring rubrics to determine clinical performance in the operating suite</title><author>Nicholson, Patricia ; Gillis, Shelley ; Dunning, A.M. Trisha</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c444t-c36528b9660a49319f4f7e0ce37843d92ef3e2633438c954b112cf50f78111f3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2009</creationdate><topic>Analysis of Variance</topic><topic>Assessment</topic><topic>Calibration</topic><topic>Career Mobility</topic><topic>Clinical Competence - standards</topic><topic>Clinical standards</topic><topic>Competence</topic><topic>Competency-based assessment</topic><topic>Construct Validity</topic><topic>Core competencies</topic><topic>Data Interpretation, Statistical</topic><topic>Educational Measurement - methods</topic><topic>Educational Measurement - standards</topic><topic>Employee Performance Appraisal - methods</topic><topic>Employee Performance Appraisal - standards</topic><topic>Humans</topic><topic>Inter-rater reliability</topic><topic>Interrater Reliability</topic><topic>Item separation index</topic><topic>Models, Nursing</topic><topic>Nurse's Role</topic><topic>Nurses</topic><topic>Nursing</topic><topic>Nursing education</topic><topic>Nursing Evaluation Research</topic><topic>Nursing Staff, Hospital - education</topic><topic>Nursing Staff, Hospital - standards</topic><topic>Observer Variation</topic><topic>Operating Room Nursing - education</topic><topic>Operating Room Nursing - standards</topic><topic>Operating theatres</topic><topic>Practice Guidelines as Topic</topic><topic>Psychometrics</topic><topic>Psychomotor Performance</topic><topic>Reliability</topic><topic>Rubrics</topic><topic>Scoring Rubrics</topic><topic>Skills</topic><topic>Validation studies</topic><topic>Validity</topic><topic>Victoria</topic><topic>Videotape Recording - methods</topic><topic>Videotape Recording - standards</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Nicholson, Patricia</creatorcontrib><creatorcontrib>Gillis, Shelley</creatorcontrib><creatorcontrib>Dunning, A.M. Trisha</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Applied Social Sciences Index & Abstracts (ASSIA)</collection><collection>British Nursing Index</collection><collection>British Nursing Index (BNI) (1985 to Present)</collection><collection>British Nursing Index</collection><collection>Nursing & Allied Health Premium</collection><collection>MEDLINE - Academic</collection><jtitle>Nurse education today</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Nicholson, Patricia</au><au>Gillis, Shelley</au><au>Dunning, A.M. Trisha</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The use of scoring rubrics to determine clinical performance in the operating suite</atitle><jtitle>Nurse education today</jtitle><addtitle>Nurse Educ Today</addtitle><date>2009</date><risdate>2009</risdate><volume>29</volume><issue>1</issue><spage>73</spage><epage>82</epage><pages>73-82</pages><issn>0260-6917</issn><eissn>1532-2793</eissn><abstract>This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.
Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.
The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (
Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.
The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.</abstract><cop>Scotland</cop><pub>Elsevier Ltd</pub><pmid>18755529</pmid><doi>10.1016/j.nedt.2008.06.011</doi><tpages>10</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0260-6917 |
| ispartof | Nurse education today, 2009, Vol.29 (1), p.73-82 |
| issn | 0260-6917 1532-2793 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_764377479 |
| source | MEDLINE; Applied Social Sciences Index & Abstracts (ASSIA); ScienceDirect Journals (5 years ago - present) |
| subjects | Analysis of Variance Assessment Calibration Career Mobility Clinical Competence - standards Clinical standards Competence Competency-based assessment Construct Validity Core competencies Data Interpretation, Statistical Educational Measurement - methods Educational Measurement - standards Employee Performance Appraisal - methods Employee Performance Appraisal - standards Humans Inter-rater reliability Interrater Reliability Item separation index Models, Nursing Nurse's Role Nurses Nursing Nursing education Nursing Evaluation Research Nursing Staff, Hospital - education Nursing Staff, Hospital - standards Observer Variation Operating Room Nursing - education Operating Room Nursing - standards Operating theatres Practice Guidelines as Topic Psychometrics Psychomotor Performance Reliability Rubrics Scoring Rubrics Skills Validation studies Validity Victoria Videotape Recording - methods Videotape Recording - standards |
| title | The use of scoring rubrics to determine clinical performance in the operating suite |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T09%3A58%3A00IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20use%20of%20scoring%20rubrics%20to%20determine%20clinical%20performance%20in%20the%20operating%20suite&rft.jtitle=Nurse%20education%20today&rft.au=Nicholson,%20Patricia&rft.date=2009&rft.volume=29&rft.issue=1&rft.spage=73&rft.epage=82&rft.pages=73-82&rft.issn=0260-6917&rft.eissn=1532-2793&rft_id=info:doi/10.1016/j.nedt.2008.06.011&rft_dat=%3Cproquest_cross%3E57264413%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=200204248&rft_id=info:pmid/18755529&rft_els_id=S0260691708000798&rfr_iscdi=true |