The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations
Background Cognizant of potential problems with validity and reliability, the Philippine Board of Surgery (PBS) undertook standardization of its oral examination procedures. This retrospective analytic study was conducted to determine and compare the reliability of the Philippine Board of Surgery or...
Gespeichert in:
| Veröffentlicht in: | Journal of surgical education 2011-03, Vol.68 (2), p.138-142 |
|---|---|
| 1. Verfasser: | |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 142 |
|---|---|
| container_issue | 2 |
| container_start_page | 138 |
| container_title | Journal of surgical education |
| container_volume | 68 |
| creator | Crisostomo, Armando C., MD |
| description | Background Cognizant of potential problems with validity and reliability, the Philippine Board of Surgery (PBS) undertook standardization of its oral examination procedures. This retrospective analytic study was conducted to determine and compare the reliability of the Philippine Board of Surgery oral examinations before and after standardization. Methods The records of oral examinations from June 29, 2003 to March 28, 2010 were reviewed and measures of reliability (intraclass correlation coefficient and kappa agreement) were computed and compared between the time periods before and after standardization and among different case content modules administered. The proportion of passers between time periods and different content areas were also compared. Results All measures of interrater reliability (intraclass correlation coefficient and kappa agreement) increased significantly overall and in all content areas and regardless of examinee outcome. There was also a trend to increased proportion of passers overall and significant improvement in passing rate in 4 out of 6 content areas. Conclusions This study validates the role of standardization of the content and scoring in improving the reliability of the oral examination, which is crucial for high stakes certifying examinations. |
| doi_str_mv | 10.1016/j.jsurg.2010.09.013 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_853473344</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S193172041000276X</els_id><sourcerecordid>853473344</sourcerecordid><originalsourceid>FETCH-LOGICAL-c413t-b39c69f7379b2f9b58608e006a156babd7a781755c1ebe95e77e82d77f2f8c653</originalsourceid><addsrcrecordid>eNqFkU9v1DAQxS0Eon_gEyCh3DhlGdtxbB9AgmoLSJWKaJG4WY4zpg7ZZLGTqsunr7NbOHCpZMn26L039m8IeUVhRYHWb7tVl-b4c8UgV0CvgPIn5JgqqUpZCfY0nzWnpWRQHZGTlDoAUWmmn5MjRjlXWrJjgtc3WKy9RzcVoy-uJju0Nrbhj53COBR5TVnwDftgm9CHabeoltLXm3zdbsOAxccxO_bu_ByMu-Iy2r5Y39lNGPYx6QV55m2f8OXDfkq-n6-vzz6XF5efvpx9uChdRflUNly7WnvJpW6Y141QNSgEqC0VdWObVlqpqBTCUWxQC5QSFWul9MwrVwt-St4ccrdx_D1jmswmJId9bwcc52SU4JXkvKqykh-ULo4pRfRmG8PGxp2hYBa8pjN7vGbBa0CbjDe7Xj_kz80G23-evzyz4N1BgPmXtwGjSS7g4LANMSM27RgeafD-P7_rwxCc7X_hDlM3znHIAA01iRkwV8uElwFTAGCy_sHvARG7ogA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>853473344</pqid></control><display><type>article</type><title>The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations</title><source>MEDLINE</source><source>Elsevier ScienceDirect Journals</source><creator>Crisostomo, Armando C., MD</creator><creatorcontrib>Crisostomo, Armando C., MD</creatorcontrib><description>Background Cognizant of potential problems with validity and reliability, the Philippine Board of Surgery (PBS) undertook standardization of its oral examination procedures. This retrospective analytic study was conducted to determine and compare the reliability of the Philippine Board of Surgery oral examinations before and after standardization. Methods The records of oral examinations from June 29, 2003 to March 28, 2010 were reviewed and measures of reliability (intraclass correlation coefficient and kappa agreement) were computed and compared between the time periods before and after standardization and among different case content modules administered. The proportion of passers between time periods and different content areas were also compared. Results All measures of interrater reliability (intraclass correlation coefficient and kappa agreement) increased significantly overall and in all content areas and regardless of examinee outcome. There was also a trend to increased proportion of passers overall and significant improvement in passing rate in 4 out of 6 content areas. Conclusions This study validates the role of standardization of the content and scoring in improving the reliability of the oral examination, which is crucial for high stakes certifying examinations.</description><identifier>ISSN: 1931-7204</identifier><identifier>EISSN: 1878-7452</identifier><identifier>DOI: 10.1016/j.jsurg.2010.09.013</identifier><identifier>PMID: 21338972</identifier><language>eng</language><publisher>United States: Elsevier Inc</publisher><subject>Accreditation - methods ; Accreditation - standards ; Confidence Intervals ; Databases, Factual ; Educational Measurement - methods ; Female ; General Surgery - education ; Humans ; inter-rater reliability ; Male ; Medical Knowledge ; Observer Variation ; oral examination ; Philippines ; reliability ; Reproducibility of Results ; Retrospective Studies ; Specialty Boards - standards ; standardization ; Surgery</subject><ispartof>Journal of surgical education, 2011-03, Vol.68 (2), p.138-142</ispartof><rights>Association of Program Directors in Surgery</rights><rights>2011 Association of Program Directors in Surgery</rights><rights>Copyright © 2011 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c413t-b39c69f7379b2f9b58608e006a156babd7a781755c1ebe95e77e82d77f2f8c653</citedby><cites>FETCH-LOGICAL-c413t-b39c69f7379b2f9b58608e006a156babd7a781755c1ebe95e77e82d77f2f8c653</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.jsurg.2010.09.013$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,778,782,3539,27907,27908,45978</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/21338972$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Crisostomo, Armando C., MD</creatorcontrib><title>The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations</title><title>Journal of surgical education</title><addtitle>J Surg Educ</addtitle><description>Background Cognizant of potential problems with validity and reliability, the Philippine Board of Surgery (PBS) undertook standardization of its oral examination procedures. This retrospective analytic study was conducted to determine and compare the reliability of the Philippine Board of Surgery oral examinations before and after standardization. Methods The records of oral examinations from June 29, 2003 to March 28, 2010 were reviewed and measures of reliability (intraclass correlation coefficient and kappa agreement) were computed and compared between the time periods before and after standardization and among different case content modules administered. The proportion of passers between time periods and different content areas were also compared. Results All measures of interrater reliability (intraclass correlation coefficient and kappa agreement) increased significantly overall and in all content areas and regardless of examinee outcome. There was also a trend to increased proportion of passers overall and significant improvement in passing rate in 4 out of 6 content areas. Conclusions This study validates the role of standardization of the content and scoring in improving the reliability of the oral examination, which is crucial for high stakes certifying examinations.</description><subject>Accreditation - methods</subject><subject>Accreditation - standards</subject><subject>Confidence Intervals</subject><subject>Databases, Factual</subject><subject>Educational Measurement - methods</subject><subject>Female</subject><subject>General Surgery - education</subject><subject>Humans</subject><subject>inter-rater reliability</subject><subject>Male</subject><subject>Medical Knowledge</subject><subject>Observer Variation</subject><subject>oral examination</subject><subject>Philippines</subject><subject>reliability</subject><subject>Reproducibility of Results</subject><subject>Retrospective Studies</subject><subject>Specialty Boards - standards</subject><subject>standardization</subject><subject>Surgery</subject><issn>1931-7204</issn><issn>1878-7452</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2011</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNqFkU9v1DAQxS0Eon_gEyCh3DhlGdtxbB9AgmoLSJWKaJG4WY4zpg7ZZLGTqsunr7NbOHCpZMn26L039m8IeUVhRYHWb7tVl-b4c8UgV0CvgPIn5JgqqUpZCfY0nzWnpWRQHZGTlDoAUWmmn5MjRjlXWrJjgtc3WKy9RzcVoy-uJju0Nrbhj53COBR5TVnwDftgm9CHabeoltLXm3zdbsOAxccxO_bu_ByMu-Iy2r5Y39lNGPYx6QV55m2f8OXDfkq-n6-vzz6XF5efvpx9uChdRflUNly7WnvJpW6Y141QNSgEqC0VdWObVlqpqBTCUWxQC5QSFWul9MwrVwt-St4ccrdx_D1jmswmJId9bwcc52SU4JXkvKqykh-ULo4pRfRmG8PGxp2hYBa8pjN7vGbBa0CbjDe7Xj_kz80G23-evzyz4N1BgPmXtwGjSS7g4LANMSM27RgeafD-P7_rwxCc7X_hDlM3znHIAA01iRkwV8uElwFTAGCy_sHvARG7ogA</recordid><startdate>20110301</startdate><enddate>20110301</enddate><creator>Crisostomo, Armando C., MD</creator><general>Elsevier Inc</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope></search><sort><creationdate>20110301</creationdate><title>The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations</title><author>Crisostomo, Armando C., MD</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c413t-b39c69f7379b2f9b58608e006a156babd7a781755c1ebe95e77e82d77f2f8c653</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2011</creationdate><topic>Accreditation - methods</topic><topic>Accreditation - standards</topic><topic>Confidence Intervals</topic><topic>Databases, Factual</topic><topic>Educational Measurement - methods</topic><topic>Female</topic><topic>General Surgery - education</topic><topic>Humans</topic><topic>inter-rater reliability</topic><topic>Male</topic><topic>Medical Knowledge</topic><topic>Observer Variation</topic><topic>oral examination</topic><topic>Philippines</topic><topic>reliability</topic><topic>Reproducibility of Results</topic><topic>Retrospective Studies</topic><topic>Specialty Boards - standards</topic><topic>standardization</topic><topic>Surgery</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Crisostomo, Armando C., MD</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Journal of surgical education</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Crisostomo, Armando C., MD</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations</atitle><jtitle>Journal of surgical education</jtitle><addtitle>J Surg Educ</addtitle><date>2011-03-01</date><risdate>2011</risdate><volume>68</volume><issue>2</issue><spage>138</spage><epage>142</epage><pages>138-142</pages><issn>1931-7204</issn><eissn>1878-7452</eissn><abstract>Background Cognizant of potential problems with validity and reliability, the Philippine Board of Surgery (PBS) undertook standardization of its oral examination procedures. This retrospective analytic study was conducted to determine and compare the reliability of the Philippine Board of Surgery oral examinations before and after standardization. Methods The records of oral examinations from June 29, 2003 to March 28, 2010 were reviewed and measures of reliability (intraclass correlation coefficient and kappa agreement) were computed and compared between the time periods before and after standardization and among different case content modules administered. The proportion of passers between time periods and different content areas were also compared. Results All measures of interrater reliability (intraclass correlation coefficient and kappa agreement) increased significantly overall and in all content areas and regardless of examinee outcome. There was also a trend to increased proportion of passers overall and significant improvement in passing rate in 4 out of 6 content areas. Conclusions This study validates the role of standardization of the content and scoring in improving the reliability of the oral examination, which is crucial for high stakes certifying examinations.</abstract><cop>United States</cop><pub>Elsevier Inc</pub><pmid>21338972</pmid><doi>10.1016/j.jsurg.2010.09.013</doi><tpages>5</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1931-7204 |
| ispartof | Journal of surgical education, 2011-03, Vol.68 (2), p.138-142 |
| issn | 1931-7204 1878-7452 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_853473344 |
| source | MEDLINE; Elsevier ScienceDirect Journals |
| subjects | Accreditation - methods Accreditation - standards Confidence Intervals Databases, Factual Educational Measurement - methods Female General Surgery - education Humans inter-rater reliability Male Medical Knowledge Observer Variation oral examination Philippines reliability Reproducibility of Results Retrospective Studies Specialty Boards - standards standardization Surgery |
| title | The Effect of Standardization on the Reliability of the Philippine Board of Surgery Oral Examinations |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-16T22%3A22%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Effect%20of%20Standardization%20on%20the%20Reliability%20of%20the%20Philippine%20Board%20of%20Surgery%20Oral%20Examinations&rft.jtitle=Journal%20of%20surgical%20education&rft.au=Crisostomo,%20Armando%20C.,%20MD&rft.date=2011-03-01&rft.volume=68&rft.issue=2&rft.spage=138&rft.epage=142&rft.pages=138-142&rft.issn=1931-7204&rft.eissn=1878-7452&rft_id=info:doi/10.1016/j.jsurg.2010.09.013&rft_dat=%3Cproquest_cross%3E853473344%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=853473344&rft_id=info:pmid/21338972&rft_els_id=S193172041000276X&rfr_iscdi=true |