Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study
Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users. Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediat...
Gespeichert in:
| Veröffentlicht in: | Prehospital and disaster medicine 2022-06, Vol.37 (3), p.306-313 |
|---|---|
| Hauptverfasser: | , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 313 |
|---|---|
| container_issue | 3 |
| container_start_page | 306 |
| container_title | Prehospital and disaster medicine |
| container_volume | 37 |
| creator | Fisher, Emilia H. Claudius, Ilene Kaji, Amy H. Shaban, Alaa McGlynn, Nicholas Cicero, Mark X. Santillanes, Genevieve Gausche-Hill, Marianne Chang, Todd P. Donofrio-Odmann, J. Joelle |
| description | Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users.
Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediatric population.
A dataset of 253 pediatric ( |
| doi_str_mv | 10.1017/S1049023X22000632 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2652863766</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cupid>10_1017_S1049023X22000632</cupid><sourcerecordid>2665523324</sourcerecordid><originalsourceid>FETCH-LOGICAL-c373t-333cf1276cecd17588b6260b0f14c0902599edc7eb679014df15a82025d403753</originalsourceid><addsrcrecordid>eNp1kUtP3DAUha0KVB7tD-imssSmmxQ_YifpbjSldCRQRwNI3UU39s1glAfYzmIW_PcaGECiYuNr-Xz3-OoeQr5w9p0zXhxfcJZXTMi_QjDGtBQfyD6vcpXxSpY76Z7k7EHfIwch3DAmKiX0R7InVZ5zVZb75H4xRPTZCtJJV9g5aFzn4obCYOls7RF7HCKd9eOwpucQQjaHMEGXiMVgnH0Uu_XoXbzuA70KLnFAl2gdRO8MvfQw9UB_QoSA8Qed0aXrxkgv4mQ3n8huC13Az9t6SK5-nVzOf2dnf04X89lZZmQhYyalNC0XhTZoLC_S4I0WmjWs5blhaQOqqtCaAhtdVIzntuUKSpHebc5koeQh-fbke-vHuwlDrHsXDHYdDDhOoRZaiVLLQuuEHr1Bb8bJD2m6RGmlhJQiTxR_oowfQ_DY1rfe9eA3NWf1Qzb1f9mknq9b56np0b50PIeRALk1hb7xzq7x9e_3bf8BEgqXgg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2665523324</pqid></control><display><type>article</type><title>Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study</title><source>MEDLINE</source><source>Cambridge University Press Journals Complete</source><creator>Fisher, Emilia H. ; Claudius, Ilene ; Kaji, Amy H. ; Shaban, Alaa ; McGlynn, Nicholas ; Cicero, Mark X. ; Santillanes, Genevieve ; Gausche-Hill, Marianne ; Chang, Todd P. ; Donofrio-Odmann, J. Joelle</creator><creatorcontrib>Fisher, Emilia H. ; Claudius, Ilene ; Kaji, Amy H. ; Shaban, Alaa ; McGlynn, Nicholas ; Cicero, Mark X. ; Santillanes, Genevieve ; Gausche-Hill, Marianne ; Chang, Todd P. ; Donofrio-Odmann, J. Joelle</creatorcontrib><description>Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users.
Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediatric population.
A dataset of 253 pediatric (<14 years of age) trauma activations from a Level I trauma center was used to obtain prehospital information and demographics. Three raters were trained on five MCI triage algorithms: Simple Triage and Rapid Treatment (START) and JumpSTART, as appropriate for age (combined as J-START); Sort Assess Life-Saving Intervention Treatment (SALT); Pediatric Triage Tape (PTT); CareFlight (CF); and Sacco Triage Method (STM). Patient outcomes were collected but not available to raters. Each rater triaged the full set of patients into Green, Yellow, Red, or Black categories with each of the five MCI algorithms. The IRR was reported as weighted kappa scores with 95% confidence intervals (CI). Descriptive statistics were used to describe inter-rater and inter-MCI algorithm agreement.
Of the 253 patients, 247 had complete triage assignments among the five algorithms and were included in the study. The IRR was excellent for a majority of the algorithms; however, J-START and CF had the highest reliability with a kappa 0.94 or higher (0.9-1.0, 95% CI for overall weighted kappa). The greatest variability was in SALT among Green and Yellow patients. Overall, J-START and CF had the highest inter-rater and inter-MCI algorithm agreements.
The IRR was excellent for a majority of the algorithms. The SALT algorithm, which contains subjective components, had the lowest IRR when applied to this dataset of pediatric trauma patients. Both J-START and CF demonstrated the best overall reliability and agreement.</description><identifier>ISSN: 1049-023X</identifier><identifier>EISSN: 1945-1938</identifier><identifier>DOI: 10.1017/S1049023X22000632</identifier><identifier>PMID: 35441588</identifier><language>eng</language><publisher>New York, USA: Cambridge University Press</publisher><subject>Agreements ; Algorithms ; Child ; Datasets ; Disasters ; Emergency medical care ; Humans ; Mass Casualty Incidents ; Original Research ; Pediatrics ; Pilot Projects ; Reproducibility of Results ; Salts ; Triage - methods</subject><ispartof>Prehospital and disaster medicine, 2022-06, Vol.37 (3), p.306-313</ispartof><rights>The Author(s), 2022. Published by Cambridge University Press on behalf of the World Association for Disaster and Emergency Medicine</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c373t-333cf1276cecd17588b6260b0f14c0902599edc7eb679014df15a82025d403753</citedby><cites>FETCH-LOGICAL-c373t-333cf1276cecd17588b6260b0f14c0902599edc7eb679014df15a82025d403753</cites><orcidid>0000-0002-2342-0521 ; 0000-0002-3292-9696 ; 0000-0002-4508-2551</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.cambridge.org/core/product/identifier/S1049023X22000632/type/journal_article$$EHTML$$P50$$Gcambridge$$H</linktohtml><link.rule.ids>164,315,781,785,27929,27930,55633</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/35441588$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Fisher, Emilia H.</creatorcontrib><creatorcontrib>Claudius, Ilene</creatorcontrib><creatorcontrib>Kaji, Amy H.</creatorcontrib><creatorcontrib>Shaban, Alaa</creatorcontrib><creatorcontrib>McGlynn, Nicholas</creatorcontrib><creatorcontrib>Cicero, Mark X.</creatorcontrib><creatorcontrib>Santillanes, Genevieve</creatorcontrib><creatorcontrib>Gausche-Hill, Marianne</creatorcontrib><creatorcontrib>Chang, Todd P.</creatorcontrib><creatorcontrib>Donofrio-Odmann, J. Joelle</creatorcontrib><title>Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study</title><title>Prehospital and disaster medicine</title><addtitle>Prehosp. Disaster med</addtitle><description>Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users.
Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediatric population.
A dataset of 253 pediatric (<14 years of age) trauma activations from a Level I trauma center was used to obtain prehospital information and demographics. Three raters were trained on five MCI triage algorithms: Simple Triage and Rapid Treatment (START) and JumpSTART, as appropriate for age (combined as J-START); Sort Assess Life-Saving Intervention Treatment (SALT); Pediatric Triage Tape (PTT); CareFlight (CF); and Sacco Triage Method (STM). Patient outcomes were collected but not available to raters. Each rater triaged the full set of patients into Green, Yellow, Red, or Black categories with each of the five MCI algorithms. The IRR was reported as weighted kappa scores with 95% confidence intervals (CI). Descriptive statistics were used to describe inter-rater and inter-MCI algorithm agreement.
Of the 253 patients, 247 had complete triage assignments among the five algorithms and were included in the study. The IRR was excellent for a majority of the algorithms; however, J-START and CF had the highest reliability with a kappa 0.94 or higher (0.9-1.0, 95% CI for overall weighted kappa). The greatest variability was in SALT among Green and Yellow patients. Overall, J-START and CF had the highest inter-rater and inter-MCI algorithm agreements.
The IRR was excellent for a majority of the algorithms. The SALT algorithm, which contains subjective components, had the lowest IRR when applied to this dataset of pediatric trauma patients. Both J-START and CF demonstrated the best overall reliability and agreement.</description><subject>Agreements</subject><subject>Algorithms</subject><subject>Child</subject><subject>Datasets</subject><subject>Disasters</subject><subject>Emergency medical care</subject><subject>Humans</subject><subject>Mass Casualty Incidents</subject><subject>Original Research</subject><subject>Pediatrics</subject><subject>Pilot Projects</subject><subject>Reproducibility of Results</subject><subject>Salts</subject><subject>Triage - methods</subject><issn>1049-023X</issn><issn>1945-1938</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><recordid>eNp1kUtP3DAUha0KVB7tD-imssSmmxQ_YifpbjSldCRQRwNI3UU39s1glAfYzmIW_PcaGECiYuNr-Xz3-OoeQr5w9p0zXhxfcJZXTMi_QjDGtBQfyD6vcpXxSpY76Z7k7EHfIwch3DAmKiX0R7InVZ5zVZb75H4xRPTZCtJJV9g5aFzn4obCYOls7RF7HCKd9eOwpucQQjaHMEGXiMVgnH0Uu_XoXbzuA70KLnFAl2gdRO8MvfQw9UB_QoSA8Qed0aXrxkgv4mQ3n8huC13Az9t6SK5-nVzOf2dnf04X89lZZmQhYyalNC0XhTZoLC_S4I0WmjWs5blhaQOqqtCaAhtdVIzntuUKSpHebc5koeQh-fbke-vHuwlDrHsXDHYdDDhOoRZaiVLLQuuEHr1Bb8bJD2m6RGmlhJQiTxR_oowfQ_DY1rfe9eA3NWf1Qzb1f9mknq9b56np0b50PIeRALk1hb7xzq7x9e_3bf8BEgqXgg</recordid><startdate>202206</startdate><enddate>202206</enddate><creator>Fisher, Emilia H.</creator><creator>Claudius, Ilene</creator><creator>Kaji, Amy H.</creator><creator>Shaban, Alaa</creator><creator>McGlynn, Nicholas</creator><creator>Cicero, Mark X.</creator><creator>Santillanes, Genevieve</creator><creator>Gausche-Hill, Marianne</creator><creator>Chang, Todd P.</creator><creator>Donofrio-Odmann, J. Joelle</creator><general>Cambridge University Press</general><general>Jems Publishing Company, Inc</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7RV</scope><scope>7X7</scope><scope>7XB</scope><scope>88C</scope><scope>88E</scope><scope>8C1</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>K9.</scope><scope>KB0</scope><scope>M0S</scope><scope>M0T</scope><scope>M1P</scope><scope>NAPCQ</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-2342-0521</orcidid><orcidid>https://orcid.org/0000-0002-3292-9696</orcidid><orcidid>https://orcid.org/0000-0002-4508-2551</orcidid></search><sort><creationdate>202206</creationdate><title>Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study</title><author>Fisher, Emilia H. ; Claudius, Ilene ; Kaji, Amy H. ; Shaban, Alaa ; McGlynn, Nicholas ; Cicero, Mark X. ; Santillanes, Genevieve ; Gausche-Hill, Marianne ; Chang, Todd P. ; Donofrio-Odmann, J. Joelle</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c373t-333cf1276cecd17588b6260b0f14c0902599edc7eb679014df15a82025d403753</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Agreements</topic><topic>Algorithms</topic><topic>Child</topic><topic>Datasets</topic><topic>Disasters</topic><topic>Emergency medical care</topic><topic>Humans</topic><topic>Mass Casualty Incidents</topic><topic>Original Research</topic><topic>Pediatrics</topic><topic>Pilot Projects</topic><topic>Reproducibility of Results</topic><topic>Salts</topic><topic>Triage - methods</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Fisher, Emilia H.</creatorcontrib><creatorcontrib>Claudius, Ilene</creatorcontrib><creatorcontrib>Kaji, Amy H.</creatorcontrib><creatorcontrib>Shaban, Alaa</creatorcontrib><creatorcontrib>McGlynn, Nicholas</creatorcontrib><creatorcontrib>Cicero, Mark X.</creatorcontrib><creatorcontrib>Santillanes, Genevieve</creatorcontrib><creatorcontrib>Gausche-Hill, Marianne</creatorcontrib><creatorcontrib>Chang, Todd P.</creatorcontrib><creatorcontrib>Donofrio-Odmann, J. Joelle</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Nursing & Allied Health Database</collection><collection>Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Healthcare Administration Database (Alumni)</collection><collection>Medical Database (Alumni Edition)</collection><collection>Public Health Database</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Nursing & Allied Health Database (Alumni Edition)</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>Healthcare Administration Database</collection><collection>Medical Database</collection><collection>Nursing & Allied Health Premium</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>MEDLINE - Academic</collection><jtitle>Prehospital and disaster medicine</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Fisher, Emilia H.</au><au>Claudius, Ilene</au><au>Kaji, Amy H.</au><au>Shaban, Alaa</au><au>McGlynn, Nicholas</au><au>Cicero, Mark X.</au><au>Santillanes, Genevieve</au><au>Gausche-Hill, Marianne</au><au>Chang, Todd P.</au><au>Donofrio-Odmann, J. Joelle</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study</atitle><jtitle>Prehospital and disaster medicine</jtitle><addtitle>Prehosp. Disaster med</addtitle><date>2022-06</date><risdate>2022</risdate><volume>37</volume><issue>3</issue><spage>306</spage><epage>313</epage><pages>306-313</pages><issn>1049-023X</issn><eissn>1945-1938</eissn><abstract>Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users.
Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediatric population.
A dataset of 253 pediatric (<14 years of age) trauma activations from a Level I trauma center was used to obtain prehospital information and demographics. Three raters were trained on five MCI triage algorithms: Simple Triage and Rapid Treatment (START) and JumpSTART, as appropriate for age (combined as J-START); Sort Assess Life-Saving Intervention Treatment (SALT); Pediatric Triage Tape (PTT); CareFlight (CF); and Sacco Triage Method (STM). Patient outcomes were collected but not available to raters. Each rater triaged the full set of patients into Green, Yellow, Red, or Black categories with each of the five MCI algorithms. The IRR was reported as weighted kappa scores with 95% confidence intervals (CI). Descriptive statistics were used to describe inter-rater and inter-MCI algorithm agreement.
Of the 253 patients, 247 had complete triage assignments among the five algorithms and were included in the study. The IRR was excellent for a majority of the algorithms; however, J-START and CF had the highest reliability with a kappa 0.94 or higher (0.9-1.0, 95% CI for overall weighted kappa). The greatest variability was in SALT among Green and Yellow patients. Overall, J-START and CF had the highest inter-rater and inter-MCI algorithm agreements.
The IRR was excellent for a majority of the algorithms. The SALT algorithm, which contains subjective components, had the lowest IRR when applied to this dataset of pediatric trauma patients. Both J-START and CF demonstrated the best overall reliability and agreement.</abstract><cop>New York, USA</cop><pub>Cambridge University Press</pub><pmid>35441588</pmid><doi>10.1017/S1049023X22000632</doi><tpages>8</tpages><orcidid>https://orcid.org/0000-0002-2342-0521</orcidid><orcidid>https://orcid.org/0000-0002-3292-9696</orcidid><orcidid>https://orcid.org/0000-0002-4508-2551</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1049-023X |
| ispartof | Prehospital and disaster medicine, 2022-06, Vol.37 (3), p.306-313 |
| issn | 1049-023X 1945-1938 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_2652863766 |
| source | MEDLINE; Cambridge University Press Journals Complete |
| subjects | Agreements Algorithms Child Datasets Disasters Emergency medical care Humans Mass Casualty Incidents Original Research Pediatrics Pilot Projects Reproducibility of Results Salts Triage - methods |
| title | Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-11T16%3A44%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Inter-Rater%20Reliability%20and%20Agreement%20Among%20Mass-Casualty%20Incident%20Algorithms%20Using%20a%20Pediatric%20Trauma%20Dataset:%20A%20Pilot%20Study&rft.jtitle=Prehospital%20and%20disaster%20medicine&rft.au=Fisher,%20Emilia%20H.&rft.date=2022-06&rft.volume=37&rft.issue=3&rft.spage=306&rft.epage=313&rft.pages=306-313&rft.issn=1049-023X&rft.eissn=1945-1938&rft_id=info:doi/10.1017/S1049023X22000632&rft_dat=%3Cproquest_cross%3E2665523324%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2665523324&rft_id=info:pmid/35441588&rft_cupid=10_1017_S1049023X22000632&rfr_iscdi=true |