Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada
The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nur...
Gespeichert in:
| Veröffentlicht in: | The American journal of surgery 2024-09, Vol.235, p.115787, Article 115787 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | 115787 |
| container_title | The American journal of surgery |
| container_volume | 235 |
| creator | Xiong, Yingqi Spence, Richard T. Hirsch, Greg Walsh, Mark J. Neumann, Katerina |
| description | The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nurse reviewers (SCNR) and its impact on risk predictions.
SCNRs in Canada were asked to code simulated operative reports. Percent agreement and free-marginal kappa correlation were calculated. The ACS-NSQIP risk calculator was utilized to illustrate its impact on risk prediction.
Responses from 44 of 150 (29.3 %) SCNRs revealed 3 to 6 different codes chosen per case, with agreement ranging from 6.7 % to 62.3 %. Free-marginal kappa correlation ranged from moderate agreement (0.53) to high disagreement (−0.17). ACS-NSQIP risk calculator predicted large absolute differences in risk for serious complications (0.2 %–13.7 %) and mortality (0.2 %–6.3 %).
This study demonstrated low inter-rater reliability in coding ACS-NSQIP colorectal procedures in Canada among SCNRs, impacting risk predictions.
•This study demonstrated low inter-rater reliability across all ACS-NSQIP colorectal resection procedure coding in Canada among SCNRs.•Coding inconsistencies resulted in significant variation in predicted morbidity and mortality, when using the NSQIP risk prediction calculator.•These results highlight the importance of ongoing efforts to improve coding standardization and education among healthcare professionals. |
| doi_str_mv | 10.1016/j.amjsurg.2024.115787 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_3073713684</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0002961024003155</els_id><sourcerecordid>3073713684</sourcerecordid><originalsourceid>FETCH-LOGICAL-c388t-2e5d6e7756679fe78ac4f983e017214fd8617a4a8967bf42241442d55297a62e3</originalsourceid><addsrcrecordid>eNqFkEtLxDAQgIMo7vr4CUrBi5eumSRN0pPI4mNFfKCeQ7adLindVpNW2H9vll09ePEywwzfPPgIOQE6AQryop7YZR0Gv5gwysQEIFNa7ZAxaJWnoDXfJWNKKUtzCXREDkKoYwkg-D4ZcZ0LIZkYk_tZ26NPvY0x8dg4O3eN61dJVyVX09f08fVl9pwUXdN5LHrbJB--K7AcPMZm6dpF4tpkaltb2iOyV9km4PE2H5L3m-u36V368HQ7m149pAXXuk8ZZqVEpTIpVV6h0rYQVa45UlAMRFVqCcoKq3Op5pVgTIAQrMwylisrGfJDcr7ZG1_5HDD0ZulCgU1jW-yGYDhVXAGXWkT07A9ad4Nv43eRyjnwSEGksg1V-C4Ej5X58G5p_coANWvZpjZb2WYt22xkx7nT7fZhvsTyd-rHbgQuNwBGHV8OvQmFwzb6c2ubpuzcPye-AcqTj_Y</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3093133681</pqid></control><display><type>article</type><title>Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada</title><source>MEDLINE</source><source>Elsevier ScienceDirect Journals</source><creator>Xiong, Yingqi ; Spence, Richard T. ; Hirsch, Greg ; Walsh, Mark J. ; Neumann, Katerina</creator><creatorcontrib>Xiong, Yingqi ; Spence, Richard T. ; Hirsch, Greg ; Walsh, Mark J. ; Neumann, Katerina</creatorcontrib><description>The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nurse reviewers (SCNR) and its impact on risk predictions.
SCNRs in Canada were asked to code simulated operative reports. Percent agreement and free-marginal kappa correlation were calculated. The ACS-NSQIP risk calculator was utilized to illustrate its impact on risk prediction.
Responses from 44 of 150 (29.3 %) SCNRs revealed 3 to 6 different codes chosen per case, with agreement ranging from 6.7 % to 62.3 %. Free-marginal kappa correlation ranged from moderate agreement (0.53) to high disagreement (−0.17). ACS-NSQIP risk calculator predicted large absolute differences in risk for serious complications (0.2 %–13.7 %) and mortality (0.2 %–6.3 %).
This study demonstrated low inter-rater reliability in coding ACS-NSQIP colorectal procedures in Canada among SCNRs, impacting risk predictions.
•This study demonstrated low inter-rater reliability across all ACS-NSQIP colorectal resection procedure coding in Canada among SCNRs.•Coding inconsistencies resulted in significant variation in predicted morbidity and mortality, when using the NSQIP risk prediction calculator.•These results highlight the importance of ongoing efforts to improve coding standardization and education among healthcare professionals.</description><identifier>ISSN: 0002-9610</identifier><identifier>ISSN: 1879-1883</identifier><identifier>EISSN: 1879-1883</identifier><identifier>DOI: 10.1016/j.amjsurg.2024.115787</identifier><identifier>PMID: 38944624</identifier><language>eng</language><publisher>United States: Elsevier Inc</publisher><subject>ACS-NSQIP ; Agreements ; Calculators ; Canada ; Clinical Coding - standards ; Codes ; Color coding ; Colorectal ; CPT code ; Current Procedural Terminology ; Humans ; Inter-rater reliability ; Laparoscopy ; Medical personnel ; Mortality ; Observer Variation ; Ostomy ; Predictions ; Provinces ; Quality control ; Quality Improvement ; Reliability ; Reproducibility of Results ; Response rates ; Risk ; Risk Assessment - methods ; Surgeons ; Surgery ; Surgical outcomes ; Terminology</subject><ispartof>The American journal of surgery, 2024-09, Vol.235, p.115787, Article 115787</ispartof><rights>2024 The Authors</rights><rights>Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.</rights><rights>2024. The Authors</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c388t-2e5d6e7756679fe78ac4f983e017214fd8617a4a8967bf42241442d55297a62e3</cites><orcidid>0000-0002-0506-0219 ; 0009-0004-9606-9121</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.sciencedirect.com/science/article/pii/S0002961024003155$$EHTML$$P50$$Gelsevier$$Hfree_for_read</linktohtml><link.rule.ids>314,776,780,3537,27901,27902,65306</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/38944624$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Xiong, Yingqi</creatorcontrib><creatorcontrib>Spence, Richard T.</creatorcontrib><creatorcontrib>Hirsch, Greg</creatorcontrib><creatorcontrib>Walsh, Mark J.</creatorcontrib><creatorcontrib>Neumann, Katerina</creatorcontrib><title>Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada</title><title>The American journal of surgery</title><addtitle>Am J Surg</addtitle><description>The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nurse reviewers (SCNR) and its impact on risk predictions.
SCNRs in Canada were asked to code simulated operative reports. Percent agreement and free-marginal kappa correlation were calculated. The ACS-NSQIP risk calculator was utilized to illustrate its impact on risk prediction.
Responses from 44 of 150 (29.3 %) SCNRs revealed 3 to 6 different codes chosen per case, with agreement ranging from 6.7 % to 62.3 %. Free-marginal kappa correlation ranged from moderate agreement (0.53) to high disagreement (−0.17). ACS-NSQIP risk calculator predicted large absolute differences in risk for serious complications (0.2 %–13.7 %) and mortality (0.2 %–6.3 %).
This study demonstrated low inter-rater reliability in coding ACS-NSQIP colorectal procedures in Canada among SCNRs, impacting risk predictions.
•This study demonstrated low inter-rater reliability across all ACS-NSQIP colorectal resection procedure coding in Canada among SCNRs.•Coding inconsistencies resulted in significant variation in predicted morbidity and mortality, when using the NSQIP risk prediction calculator.•These results highlight the importance of ongoing efforts to improve coding standardization and education among healthcare professionals.</description><subject>ACS-NSQIP</subject><subject>Agreements</subject><subject>Calculators</subject><subject>Canada</subject><subject>Clinical Coding - standards</subject><subject>Codes</subject><subject>Color coding</subject><subject>Colorectal</subject><subject>CPT code</subject><subject>Current Procedural Terminology</subject><subject>Humans</subject><subject>Inter-rater reliability</subject><subject>Laparoscopy</subject><subject>Medical personnel</subject><subject>Mortality</subject><subject>Observer Variation</subject><subject>Ostomy</subject><subject>Predictions</subject><subject>Provinces</subject><subject>Quality control</subject><subject>Quality Improvement</subject><subject>Reliability</subject><subject>Reproducibility of Results</subject><subject>Response rates</subject><subject>Risk</subject><subject>Risk Assessment - methods</subject><subject>Surgeons</subject><subject>Surgery</subject><subject>Surgical outcomes</subject><subject>Terminology</subject><issn>0002-9610</issn><issn>1879-1883</issn><issn>1879-1883</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>8G5</sourceid><sourceid>BENPR</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNqFkEtLxDAQgIMo7vr4CUrBi5eumSRN0pPI4mNFfKCeQ7adLindVpNW2H9vll09ePEywwzfPPgIOQE6AQryop7YZR0Gv5gwysQEIFNa7ZAxaJWnoDXfJWNKKUtzCXREDkKoYwkg-D4ZcZ0LIZkYk_tZ26NPvY0x8dg4O3eN61dJVyVX09f08fVl9pwUXdN5LHrbJB--K7AcPMZm6dpF4tpkaltb2iOyV9km4PE2H5L3m-u36V368HQ7m149pAXXuk8ZZqVEpTIpVV6h0rYQVa45UlAMRFVqCcoKq3Op5pVgTIAQrMwylisrGfJDcr7ZG1_5HDD0ZulCgU1jW-yGYDhVXAGXWkT07A9ad4Nv43eRyjnwSEGksg1V-C4Ej5X58G5p_coANWvZpjZb2WYt22xkx7nT7fZhvsTyd-rHbgQuNwBGHV8OvQmFwzb6c2ubpuzcPye-AcqTj_Y</recordid><startdate>20240901</startdate><enddate>20240901</enddate><creator>Xiong, Yingqi</creator><creator>Spence, Richard T.</creator><creator>Hirsch, Greg</creator><creator>Walsh, Mark J.</creator><creator>Neumann, Katerina</creator><general>Elsevier Inc</general><general>Elsevier Limited</general><scope>6I.</scope><scope>AAFTH</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7QO</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8FD</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>K9.</scope><scope>M0S</scope><scope>M1P</scope><scope>M2O</scope><scope>MBDVC</scope><scope>P64</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-0506-0219</orcidid><orcidid>https://orcid.org/0009-0004-9606-9121</orcidid></search><sort><creationdate>20240901</creationdate><title>Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada</title><author>Xiong, Yingqi ; Spence, Richard T. ; Hirsch, Greg ; Walsh, Mark J. ; Neumann, Katerina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c388t-2e5d6e7756679fe78ac4f983e017214fd8617a4a8967bf42241442d55297a62e3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>ACS-NSQIP</topic><topic>Agreements</topic><topic>Calculators</topic><topic>Canada</topic><topic>Clinical Coding - standards</topic><topic>Codes</topic><topic>Color coding</topic><topic>Colorectal</topic><topic>CPT code</topic><topic>Current Procedural Terminology</topic><topic>Humans</topic><topic>Inter-rater reliability</topic><topic>Laparoscopy</topic><topic>Medical personnel</topic><topic>Mortality</topic><topic>Observer Variation</topic><topic>Ostomy</topic><topic>Predictions</topic><topic>Provinces</topic><topic>Quality control</topic><topic>Quality Improvement</topic><topic>Reliability</topic><topic>Reproducibility of Results</topic><topic>Response rates</topic><topic>Risk</topic><topic>Risk Assessment - methods</topic><topic>Surgeons</topic><topic>Surgery</topic><topic>Surgical outcomes</topic><topic>Terminology</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Xiong, Yingqi</creatorcontrib><creatorcontrib>Spence, Richard T.</creatorcontrib><creatorcontrib>Hirsch, Greg</creatorcontrib><creatorcontrib>Walsh, Mark J.</creatorcontrib><creatorcontrib>Neumann, Katerina</creatorcontrib><collection>ScienceDirect Open Access Titles</collection><collection>Elsevier:ScienceDirect:Open Access</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Biotechnology Research Abstracts</collection><collection>ProQuest Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>Technology Research Database</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>PML(ProQuest Medical Library)</collection><collection>ProQuest Research Library</collection><collection>Research Library (Corporate)</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>MEDLINE - Academic</collection><jtitle>The American journal of surgery</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xiong, Yingqi</au><au>Spence, Richard T.</au><au>Hirsch, Greg</au><au>Walsh, Mark J.</au><au>Neumann, Katerina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada</atitle><jtitle>The American journal of surgery</jtitle><addtitle>Am J Surg</addtitle><date>2024-09-01</date><risdate>2024</risdate><volume>235</volume><spage>115787</spage><pages>115787-</pages><artnum>115787</artnum><issn>0002-9610</issn><issn>1879-1883</issn><eissn>1879-1883</eissn><abstract>The American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) uses Current Procedural Terminology (CPT) codes for risk-adjusted calculations. This study evaluates the inter-rater reliability of coding colorectal resections across Canada by ACS-NSQIP surgical clinical nurse reviewers (SCNR) and its impact on risk predictions.
SCNRs in Canada were asked to code simulated operative reports. Percent agreement and free-marginal kappa correlation were calculated. The ACS-NSQIP risk calculator was utilized to illustrate its impact on risk prediction.
Responses from 44 of 150 (29.3 %) SCNRs revealed 3 to 6 different codes chosen per case, with agreement ranging from 6.7 % to 62.3 %. Free-marginal kappa correlation ranged from moderate agreement (0.53) to high disagreement (−0.17). ACS-NSQIP risk calculator predicted large absolute differences in risk for serious complications (0.2 %–13.7 %) and mortality (0.2 %–6.3 %).
This study demonstrated low inter-rater reliability in coding ACS-NSQIP colorectal procedures in Canada among SCNRs, impacting risk predictions.
•This study demonstrated low inter-rater reliability across all ACS-NSQIP colorectal resection procedure coding in Canada among SCNRs.•Coding inconsistencies resulted in significant variation in predicted morbidity and mortality, when using the NSQIP risk prediction calculator.•These results highlight the importance of ongoing efforts to improve coding standardization and education among healthcare professionals.</abstract><cop>United States</cop><pub>Elsevier Inc</pub><pmid>38944624</pmid><doi>10.1016/j.amjsurg.2024.115787</doi><orcidid>https://orcid.org/0000-0002-0506-0219</orcidid><orcidid>https://orcid.org/0009-0004-9606-9121</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0002-9610 |
| ispartof | The American journal of surgery, 2024-09, Vol.235, p.115787, Article 115787 |
| issn | 0002-9610 1879-1883 1879-1883 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_3073713684 |
| source | MEDLINE; Elsevier ScienceDirect Journals |
| subjects | ACS-NSQIP Agreements Calculators Canada Clinical Coding - standards Codes Color coding Colorectal CPT code Current Procedural Terminology Humans Inter-rater reliability Laparoscopy Medical personnel Mortality Observer Variation Ostomy Predictions Provinces Quality control Quality Improvement Reliability Reproducibility of Results Response rates Risk Risk Assessment - methods Surgeons Surgery Surgical outcomes Terminology |
| title | Inter-rater reliability of ACS-NSQIP colorectal procedure coding in Canada |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-12T00%3A03%3A26IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Inter-rater%20reliability%20of%20ACS-NSQIP%20colorectal%20procedure%20coding%20in%20Canada&rft.jtitle=The%20American%20journal%20of%20surgery&rft.au=Xiong,%20Yingqi&rft.date=2024-09-01&rft.volume=235&rft.spage=115787&rft.pages=115787-&rft.artnum=115787&rft.issn=0002-9610&rft.eissn=1879-1883&rft_id=info:doi/10.1016/j.amjsurg.2024.115787&rft_dat=%3Cproquest_cross%3E3073713684%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3093133681&rft_id=info:pmid/38944624&rft_els_id=S0002961024003155&rfr_iscdi=true |