Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course
Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall O...
Gespeichert in:
| Veröffentlicht in: | Currents in pharmacy teaching and learning 2024-11, Vol.16 (11), p.102159, Article 102159 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | 11 |
| container_start_page | 102159 |
| container_title | Currents in pharmacy teaching and learning |
| container_volume | 16 |
| creator | Bowers, Riley D. Baker, Carrie N. Becker, Kaitlyn K. Hamilton, Jessica N. Trotta, Katie |
| description | Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall OSCE scores compared between faculty, peer, and self-evaluations within a Doctor of Pharmacy (PharmD) curriculum.
This study was conducted during the required nonprescription therapeutics course. Seventy-seven first-year PharmD students were included in the study, with 6 faculty members grading 10–15 students each. Students were evaluated by 3 graders: self, peer, and faculty. All evaluators utilized the same rubric. The primary endpoint of the study was to compare the overall scores between groups. Secondary endpoints included interrater reliability and quantification of feedback type based on the evaluator group.
The maximum possible score for the OSCE was 50 points; the mean scores for self, peer, and faculty evaluations were 43.3, 43.5, and 41.7 points, respectively. No statistically significant difference was found between the self and peer raters. However, statistical significance was found in the comparison of self versus faculty (p = 0.005) and in peer versus faculty (p |
| doi_str_mv | 10.1016/j.cptl.2024.102159 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_3087353257</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S1877129724001916</els_id><sourcerecordid>3087353257</sourcerecordid><originalsourceid>FETCH-LOGICAL-c237t-9e873d9249f939c9312716c61cca9d302d5bc5333a3d406a8042bd4c6c9863303</originalsourceid><addsrcrecordid>eNp9kctuFDEQRS1ERB7wAyyQlywyEz_6ZYkNGiAgRQoLWFuecrXikdtu_Bgl38BPpyeTsGRVV6VTt1R1CXnP2Zoz3l3t1jAXvxZMNEtD8Fa9Imd86PsVl4y9ftFC9afkPOcdYz1jnXxDTqVigxJ8OCN_N3GaTXI5BhpHOiOmS5rRj5fUBEtHA9WXBxq3O4Ti9khzSRVKTWgpeBccGE_x3kwumOIWE9wbX59kpi5QQ3_emTR9oSGGOWGG5OYnrtxhMjPW4iBTiDVlfEtORuMzvnuuF-T3t6-_Nt9XN7fXPzafb1YgZF9WCodeWiUaNSqpQEkuet5BxwGMspIJ226hlVIaaRvWmYE1Ymsb6EANnZRMXpCPR985xT8Vc9GTy4Dem4CxZi3ZsqCVou0XVBxRSDHnhKOek5tMetCc6UMIeqcPIehDCPoYwjL04dm_bie0_0Zevr4An44ALlfuHSadwWEAtC4tb9Y2uv_5PwKkaJrJ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3087353257</pqid></control><display><type>article</type><title>Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course</title><source>MEDLINE</source><source>ScienceDirect Journals (5 years ago - present)</source><creator>Bowers, Riley D. ; Baker, Carrie N. ; Becker, Kaitlyn K. ; Hamilton, Jessica N. ; Trotta, Katie</creator><creatorcontrib>Bowers, Riley D. ; Baker, Carrie N. ; Becker, Kaitlyn K. ; Hamilton, Jessica N. ; Trotta, Katie</creatorcontrib><description>Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall OSCE scores compared between faculty, peer, and self-evaluations within a Doctor of Pharmacy (PharmD) curriculum.
This study was conducted during the required nonprescription therapeutics course. Seventy-seven first-year PharmD students were included in the study, with 6 faculty members grading 10–15 students each. Students were evaluated by 3 graders: self, peer, and faculty. All evaluators utilized the same rubric. The primary endpoint of the study was to compare the overall scores between groups. Secondary endpoints included interrater reliability and quantification of feedback type based on the evaluator group.
The maximum possible score for the OSCE was 50 points; the mean scores for self, peer, and faculty evaluations were 43.3, 43.5, and 41.7 points, respectively. No statistically significant difference was found between the self and peer raters. However, statistical significance was found in the comparison of self versus faculty (p = 0.005) and in peer versus faculty (p < 0.001). When these scores were correlated to a letter grade (A, B, C or less), higher grades had greater similarity among raters compared to lower scores. Despite differences in scoring, the interrater reliability, or W score, on overall letter grade was 0.79, which is considered strong agreement.
This study successfully demonstrated how peer and self-evaluation of an OSCE provides a comparable alternative to traditional faculty grading, especially in higher performing students. However, due to differences in overall grades, this strategy should be reserved for low-stakes assessments and basic skill evaluations.</description><identifier>ISSN: 1877-1297</identifier><identifier>ISSN: 1877-1300</identifier><identifier>EISSN: 1877-1300</identifier><identifier>DOI: 10.1016/j.cptl.2024.102159</identifier><identifier>PMID: 39089218</identifier><language>eng</language><publisher>United States: Elsevier Inc</publisher><subject>Clinical Competence - standards ; Clinical Competence - statistics & numerical data ; Curriculum - standards ; Curriculum - trends ; Education, Pharmacy - methods ; Education, Pharmacy - standards ; Education, Pharmacy - statistics & numerical data ; Educational Measurement - methods ; Educational Measurement - standards ; Educational Measurement - statistics & numerical data ; Faculty, Pharmacy - statistics & numerical data ; Female ; Humans ; Male ; OSCE ; Peer Group ; Peer-evaluation ; Reproducibility of Results ; Rubrics ; Self-Assessment ; Self-evaluation ; Standardized patient ; Students, Pharmacy - psychology ; Students, Pharmacy - statistics & numerical data</subject><ispartof>Currents in pharmacy teaching and learning, 2024-11, Vol.16 (11), p.102159, Article 102159</ispartof><rights>2024</rights><rights>Copyright © 2024. Published by Elsevier Inc.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c237t-9e873d9249f939c9312716c61cca9d302d5bc5333a3d406a8042bd4c6c9863303</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.cptl.2024.102159$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,780,784,3550,27924,27925,45995</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/39089218$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Bowers, Riley D.</creatorcontrib><creatorcontrib>Baker, Carrie N.</creatorcontrib><creatorcontrib>Becker, Kaitlyn K.</creatorcontrib><creatorcontrib>Hamilton, Jessica N.</creatorcontrib><creatorcontrib>Trotta, Katie</creatorcontrib><title>Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course</title><title>Currents in pharmacy teaching and learning</title><addtitle>Curr Pharm Teach Learn</addtitle><description>Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall OSCE scores compared between faculty, peer, and self-evaluations within a Doctor of Pharmacy (PharmD) curriculum.
This study was conducted during the required nonprescription therapeutics course. Seventy-seven first-year PharmD students were included in the study, with 6 faculty members grading 10–15 students each. Students were evaluated by 3 graders: self, peer, and faculty. All evaluators utilized the same rubric. The primary endpoint of the study was to compare the overall scores between groups. Secondary endpoints included interrater reliability and quantification of feedback type based on the evaluator group.
The maximum possible score for the OSCE was 50 points; the mean scores for self, peer, and faculty evaluations were 43.3, 43.5, and 41.7 points, respectively. No statistically significant difference was found between the self and peer raters. However, statistical significance was found in the comparison of self versus faculty (p = 0.005) and in peer versus faculty (p < 0.001). When these scores were correlated to a letter grade (A, B, C or less), higher grades had greater similarity among raters compared to lower scores. Despite differences in scoring, the interrater reliability, or W score, on overall letter grade was 0.79, which is considered strong agreement.
This study successfully demonstrated how peer and self-evaluation of an OSCE provides a comparable alternative to traditional faculty grading, especially in higher performing students. However, due to differences in overall grades, this strategy should be reserved for low-stakes assessments and basic skill evaluations.</description><subject>Clinical Competence - standards</subject><subject>Clinical Competence - statistics & numerical data</subject><subject>Curriculum - standards</subject><subject>Curriculum - trends</subject><subject>Education, Pharmacy - methods</subject><subject>Education, Pharmacy - standards</subject><subject>Education, Pharmacy - statistics & numerical data</subject><subject>Educational Measurement - methods</subject><subject>Educational Measurement - standards</subject><subject>Educational Measurement - statistics & numerical data</subject><subject>Faculty, Pharmacy - statistics & numerical data</subject><subject>Female</subject><subject>Humans</subject><subject>Male</subject><subject>OSCE</subject><subject>Peer Group</subject><subject>Peer-evaluation</subject><subject>Reproducibility of Results</subject><subject>Rubrics</subject><subject>Self-Assessment</subject><subject>Self-evaluation</subject><subject>Standardized patient</subject><subject>Students, Pharmacy - psychology</subject><subject>Students, Pharmacy - statistics & numerical data</subject><issn>1877-1297</issn><issn>1877-1300</issn><issn>1877-1300</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp9kctuFDEQRS1ERB7wAyyQlywyEz_6ZYkNGiAgRQoLWFuecrXikdtu_Bgl38BPpyeTsGRVV6VTt1R1CXnP2Zoz3l3t1jAXvxZMNEtD8Fa9Imd86PsVl4y9ftFC9afkPOcdYz1jnXxDTqVigxJ8OCN_N3GaTXI5BhpHOiOmS5rRj5fUBEtHA9WXBxq3O4Ti9khzSRVKTWgpeBccGE_x3kwumOIWE9wbX59kpi5QQ3_emTR9oSGGOWGG5OYnrtxhMjPW4iBTiDVlfEtORuMzvnuuF-T3t6-_Nt9XN7fXPzafb1YgZF9WCodeWiUaNSqpQEkuet5BxwGMspIJ226hlVIaaRvWmYE1Ymsb6EANnZRMXpCPR985xT8Vc9GTy4Dem4CxZi3ZsqCVou0XVBxRSDHnhKOek5tMetCc6UMIeqcPIehDCPoYwjL04dm_bie0_0Zevr4An44ALlfuHSadwWEAtC4tb9Y2uv_5PwKkaJrJ</recordid><startdate>202411</startdate><enddate>202411</enddate><creator>Bowers, Riley D.</creator><creator>Baker, Carrie N.</creator><creator>Becker, Kaitlyn K.</creator><creator>Hamilton, Jessica N.</creator><creator>Trotta, Katie</creator><general>Elsevier Inc</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope></search><sort><creationdate>202411</creationdate><title>Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course</title><author>Bowers, Riley D. ; Baker, Carrie N. ; Becker, Kaitlyn K. ; Hamilton, Jessica N. ; Trotta, Katie</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c237t-9e873d9249f939c9312716c61cca9d302d5bc5333a3d406a8042bd4c6c9863303</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Clinical Competence - standards</topic><topic>Clinical Competence - statistics & numerical data</topic><topic>Curriculum - standards</topic><topic>Curriculum - trends</topic><topic>Education, Pharmacy - methods</topic><topic>Education, Pharmacy - standards</topic><topic>Education, Pharmacy - statistics & numerical data</topic><topic>Educational Measurement - methods</topic><topic>Educational Measurement - standards</topic><topic>Educational Measurement - statistics & numerical data</topic><topic>Faculty, Pharmacy - statistics & numerical data</topic><topic>Female</topic><topic>Humans</topic><topic>Male</topic><topic>OSCE</topic><topic>Peer Group</topic><topic>Peer-evaluation</topic><topic>Reproducibility of Results</topic><topic>Rubrics</topic><topic>Self-Assessment</topic><topic>Self-evaluation</topic><topic>Standardized patient</topic><topic>Students, Pharmacy - psychology</topic><topic>Students, Pharmacy - statistics & numerical data</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Bowers, Riley D.</creatorcontrib><creatorcontrib>Baker, Carrie N.</creatorcontrib><creatorcontrib>Becker, Kaitlyn K.</creatorcontrib><creatorcontrib>Hamilton, Jessica N.</creatorcontrib><creatorcontrib>Trotta, Katie</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Currents in pharmacy teaching and learning</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Bowers, Riley D.</au><au>Baker, Carrie N.</au><au>Becker, Kaitlyn K.</au><au>Hamilton, Jessica N.</au><au>Trotta, Katie</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course</atitle><jtitle>Currents in pharmacy teaching and learning</jtitle><addtitle>Curr Pharm Teach Learn</addtitle><date>2024-11</date><risdate>2024</risdate><volume>16</volume><issue>11</issue><spage>102159</spage><pages>102159-</pages><artnum>102159</artnum><issn>1877-1297</issn><issn>1877-1300</issn><eissn>1877-1300</eissn><abstract>Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall OSCE scores compared between faculty, peer, and self-evaluations within a Doctor of Pharmacy (PharmD) curriculum.
This study was conducted during the required nonprescription therapeutics course. Seventy-seven first-year PharmD students were included in the study, with 6 faculty members grading 10–15 students each. Students were evaluated by 3 graders: self, peer, and faculty. All evaluators utilized the same rubric. The primary endpoint of the study was to compare the overall scores between groups. Secondary endpoints included interrater reliability and quantification of feedback type based on the evaluator group.
The maximum possible score for the OSCE was 50 points; the mean scores for self, peer, and faculty evaluations were 43.3, 43.5, and 41.7 points, respectively. No statistically significant difference was found between the self and peer raters. However, statistical significance was found in the comparison of self versus faculty (p = 0.005) and in peer versus faculty (p < 0.001). When these scores were correlated to a letter grade (A, B, C or less), higher grades had greater similarity among raters compared to lower scores. Despite differences in scoring, the interrater reliability, or W score, on overall letter grade was 0.79, which is considered strong agreement.
This study successfully demonstrated how peer and self-evaluation of an OSCE provides a comparable alternative to traditional faculty grading, especially in higher performing students. However, due to differences in overall grades, this strategy should be reserved for low-stakes assessments and basic skill evaluations.</abstract><cop>United States</cop><pub>Elsevier Inc</pub><pmid>39089218</pmid><doi>10.1016/j.cptl.2024.102159</doi></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1877-1297 |
| ispartof | Currents in pharmacy teaching and learning, 2024-11, Vol.16 (11), p.102159, Article 102159 |
| issn | 1877-1297 1877-1300 1877-1300 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_3087353257 |
| source | MEDLINE; ScienceDirect Journals (5 years ago - present) |
| subjects | Clinical Competence - standards Clinical Competence - statistics & numerical data Curriculum - standards Curriculum - trends Education, Pharmacy - methods Education, Pharmacy - standards Education, Pharmacy - statistics & numerical data Educational Measurement - methods Educational Measurement - standards Educational Measurement - statistics & numerical data Faculty, Pharmacy - statistics & numerical data Female Humans Male OSCE Peer Group Peer-evaluation Reproducibility of Results Rubrics Self-Assessment Self-evaluation Standardized patient Students, Pharmacy - psychology Students, Pharmacy - statistics & numerical data |
| title | Comparison of peer, self, and faculty objective structured clinical examination evaluations in a PharmD nonprescription therapeutics course |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T03%3A34%3A31IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Comparison%20of%20peer,%20self,%20and%20faculty%20objective%20structured%20clinical%20examination%20evaluations%20in%20a%20PharmD%20nonprescription%20therapeutics%20course&rft.jtitle=Currents%20in%20pharmacy%20teaching%20and%20learning&rft.au=Bowers,%20Riley%20D.&rft.date=2024-11&rft.volume=16&rft.issue=11&rft.spage=102159&rft.pages=102159-&rft.artnum=102159&rft.issn=1877-1297&rft.eissn=1877-1300&rft_id=info:doi/10.1016/j.cptl.2024.102159&rft_dat=%3Cproquest_cross%3E3087353257%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3087353257&rft_id=info:pmid/39089218&rft_els_id=S1877129724001916&rfr_iscdi=true |