The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield
Objective Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high‐yield detection of harm, consi...
Gespeichert in:
| Veröffentlicht in: | Academic emergency medicine 2020-12, Vol.27 (12), p.1279-1290 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 1290 |
|---|---|
| container_issue | 12 |
| container_start_page | 1279 |
| container_title | Academic emergency medicine |
| container_volume | 27 |
| creator | Griffey, Richard T. Schneider, Ryan M. Todorov, Alexandre A. Venkatesh, Arjun K. |
| description | Objective
Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high‐yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement.
Methods
This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two‐tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts).
Results
Twenty‐four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p |
| doi_str_mv | 10.1111/acem.14101 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2430373720</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2471156798</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3931-31dc98c73d9e535c88ec043967ce1386a4c8db5023c20c7cac4f433fe4e01f353</originalsourceid><addsrcrecordid>eNp90E1Lw0AQBuBFFFurF3-ALHgRIXU3s5tNvJVaP0DpJQqewnYzqVvyUTcpUn-9W1s9eHAuM4eHl-El5JSzIfdzpQ1WQy4443ukz6WEIFQ83Pc3i5IgkhH0yFHbLhhjUiXqkPQgVEKGseiTafqGdFKhm2Nt1vQGl9p1FdYdTZ2dz9HRtGnKa_qiS5vrzjY11XVOU2w7W89p19DpsrOV_UT6arHMj8lBocsWT3Z7QJ5vJ-n4Pnic3j2MR4-BgQR4ADw3SWwU5AlKkCaO0TABSaQMcogjLUyczyQLwYTMKKONKARAgQIZL0DCgFxsc5eueV_5b7LKtgbLUtfYrNosFMBAgQqZp-d_6KJZudp_55XiXEYqib263CrjmrZ1WGRLZyvt1hln2abmbFNz9l2zx2e7yNWswvyX_vTqAd-CD1vi-p-obDSePG1DvwAyiYW0</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2471156798</pqid></control><display><type>article</type><title>The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield</title><source>MEDLINE</source><source>Access via Wiley Online Library</source><source>EZB-FREE-00999 freely available EZB journals</source><source>Wiley Online Library (Open Access Collection)</source><creator>Griffey, Richard T. ; Schneider, Ryan M. ; Todorov, Alexandre A. ; Venkatesh, Arjun K.</creator><contributor>Venkatesh, Arjun K.</contributor><creatorcontrib>Griffey, Richard T. ; Schneider, Ryan M. ; Todorov, Alexandre A. ; Venkatesh, Arjun K. ; Venkatesh, Arjun K.</creatorcontrib><description>Objective
Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high‐yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement.
Methods
This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two‐tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts).
Results
Twenty‐four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p < 0.001). The risk of an AE increased with number of triggers. Combining data from both phases, we identified 461 AEs in 429 unique visits in 5,582 records reviewed. Our multivariable model (which emphasized parsimony) retained 12 triggers with a ROC AUC of 82% in both samples. Selecting records for review based on number of triggers improves yield to 14% for 4+ triggers (top 10% of visits) and to 28% for 8+ (top 1%). A weighted trigger count has corresponding yields of 18 and 38%. The method for selecting records for review did not appear to affect event‐type representativeness, with similar distributions of event types and severities detected.
Conclusions
In this single‐site study of the EDTT we observed high levels of validity in trigger selection, yield, and representativeness of AEs, with yields that are superior to estimates for traditional approaches to AE detection. Record selection using weighted triggers outperforms a trigger count threshold approach and far outperforms random sampling from records with at least one trigger. The EDTT is a promising efficient and high‐yield approach for detecting all‐cause harm to guide quality improvement efforts in the ED.</description><identifier>ISSN: 1069-6563</identifier><identifier>EISSN: 1553-2712</identifier><identifier>DOI: 10.1111/acem.14101</identifier><identifier>PMID: 32745284</identifier><language>eng</language><publisher>United States: Wiley Subscription Services, Inc</publisher><subject>Adolescent ; Adult ; Aged ; Electronic Health Records ; Emergency medical care ; Emergency Service, Hospital ; Humans ; Medical Errors - prevention & control ; Middle Aged ; Patient Safety ; Quality Improvement ; Retrospective Studies ; Side effects ; Vital signs ; Young Adult</subject><ispartof>Academic emergency medicine, 2020-12, Vol.27 (12), p.1279-1290</ispartof><rights>2020 by the Society for Academic Emergency Medicine</rights><rights>2020 by the Society for Academic Emergency Medicine.</rights><rights>Copyright © 2020 Society for Academic Emergency Medicine</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3931-31dc98c73d9e535c88ec043967ce1386a4c8db5023c20c7cac4f433fe4e01f353</citedby><cites>FETCH-LOGICAL-c3931-31dc98c73d9e535c88ec043967ce1386a4c8db5023c20c7cac4f433fe4e01f353</cites><orcidid>0000-0002-6279-7380 ; 0000-0001-7122-1079</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Facem.14101$$EPDF$$P50$$Gwiley$$H</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Facem.14101$$EHTML$$P50$$Gwiley$$H</linktohtml><link.rule.ids>314,780,784,1417,1433,27924,27925,45574,45575,46409,46833</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/32745284$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Venkatesh, Arjun K.</contributor><creatorcontrib>Griffey, Richard T.</creatorcontrib><creatorcontrib>Schneider, Ryan M.</creatorcontrib><creatorcontrib>Todorov, Alexandre A.</creatorcontrib><creatorcontrib>Venkatesh, Arjun K.</creatorcontrib><title>The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield</title><title>Academic emergency medicine</title><addtitle>Acad Emerg Med</addtitle><description>Objective
Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high‐yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement.
Methods
This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two‐tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts).
Results
Twenty‐four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p < 0.001). The risk of an AE increased with number of triggers. Combining data from both phases, we identified 461 AEs in 429 unique visits in 5,582 records reviewed. Our multivariable model (which emphasized parsimony) retained 12 triggers with a ROC AUC of 82% in both samples. Selecting records for review based on number of triggers improves yield to 14% for 4+ triggers (top 10% of visits) and to 28% for 8+ (top 1%). A weighted trigger count has corresponding yields of 18 and 38%. The method for selecting records for review did not appear to affect event‐type representativeness, with similar distributions of event types and severities detected.
Conclusions
In this single‐site study of the EDTT we observed high levels of validity in trigger selection, yield, and representativeness of AEs, with yields that are superior to estimates for traditional approaches to AE detection. Record selection using weighted triggers outperforms a trigger count threshold approach and far outperforms random sampling from records with at least one trigger. The EDTT is a promising efficient and high‐yield approach for detecting all‐cause harm to guide quality improvement efforts in the ED.</description><subject>Adolescent</subject><subject>Adult</subject><subject>Aged</subject><subject>Electronic Health Records</subject><subject>Emergency medical care</subject><subject>Emergency Service, Hospital</subject><subject>Humans</subject><subject>Medical Errors - prevention & control</subject><subject>Middle Aged</subject><subject>Patient Safety</subject><subject>Quality Improvement</subject><subject>Retrospective Studies</subject><subject>Side effects</subject><subject>Vital signs</subject><subject>Young Adult</subject><issn>1069-6563</issn><issn>1553-2712</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp90E1Lw0AQBuBFFFurF3-ALHgRIXU3s5tNvJVaP0DpJQqewnYzqVvyUTcpUn-9W1s9eHAuM4eHl-El5JSzIfdzpQ1WQy4443ukz6WEIFQ83Pc3i5IgkhH0yFHbLhhjUiXqkPQgVEKGseiTafqGdFKhm2Nt1vQGl9p1FdYdTZ2dz9HRtGnKa_qiS5vrzjY11XVOU2w7W89p19DpsrOV_UT6arHMj8lBocsWT3Z7QJ5vJ-n4Pnic3j2MR4-BgQR4ADw3SWwU5AlKkCaO0TABSaQMcogjLUyczyQLwYTMKKONKARAgQIZL0DCgFxsc5eueV_5b7LKtgbLUtfYrNosFMBAgQqZp-d_6KJZudp_55XiXEYqib263CrjmrZ1WGRLZyvt1hln2abmbFNz9l2zx2e7yNWswvyX_vTqAd-CD1vi-p-obDSePG1DvwAyiYW0</recordid><startdate>202012</startdate><enddate>202012</enddate><creator>Griffey, Richard T.</creator><creator>Schneider, Ryan M.</creator><creator>Todorov, Alexandre A.</creator><creator>Venkatesh, Arjun K.</creator><general>Wiley Subscription Services, Inc</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>K9.</scope><scope>NAPCQ</scope><scope>U9A</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-6279-7380</orcidid><orcidid>https://orcid.org/0000-0001-7122-1079</orcidid></search><sort><creationdate>202012</creationdate><title>The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield</title><author>Griffey, Richard T. ; Schneider, Ryan M. ; Todorov, Alexandre A. ; Venkatesh, Arjun K.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3931-31dc98c73d9e535c88ec043967ce1386a4c8db5023c20c7cac4f433fe4e01f353</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Adolescent</topic><topic>Adult</topic><topic>Aged</topic><topic>Electronic Health Records</topic><topic>Emergency medical care</topic><topic>Emergency Service, Hospital</topic><topic>Humans</topic><topic>Medical Errors - prevention & control</topic><topic>Middle Aged</topic><topic>Patient Safety</topic><topic>Quality Improvement</topic><topic>Retrospective Studies</topic><topic>Side effects</topic><topic>Vital signs</topic><topic>Young Adult</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Griffey, Richard T.</creatorcontrib><creatorcontrib>Schneider, Ryan M.</creatorcontrib><creatorcontrib>Todorov, Alexandre A.</creatorcontrib><creatorcontrib>Venkatesh, Arjun K.</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Nursing & Allied Health Premium</collection><collection>MEDLINE - Academic</collection><jtitle>Academic emergency medicine</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Griffey, Richard T.</au><au>Schneider, Ryan M.</au><au>Todorov, Alexandre A.</au><au>Venkatesh, Arjun K.</au><au>Venkatesh, Arjun K.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield</atitle><jtitle>Academic emergency medicine</jtitle><addtitle>Acad Emerg Med</addtitle><date>2020-12</date><risdate>2020</risdate><volume>27</volume><issue>12</issue><spage>1279</spage><epage>1290</epage><pages>1279-1290</pages><issn>1069-6563</issn><eissn>1553-2712</eissn><abstract>Objective
Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high‐yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement.
Methods
This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two‐tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts).
Results
Twenty‐four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p < 0.001). The risk of an AE increased with number of triggers. Combining data from both phases, we identified 461 AEs in 429 unique visits in 5,582 records reviewed. Our multivariable model (which emphasized parsimony) retained 12 triggers with a ROC AUC of 82% in both samples. Selecting records for review based on number of triggers improves yield to 14% for 4+ triggers (top 10% of visits) and to 28% for 8+ (top 1%). A weighted trigger count has corresponding yields of 18 and 38%. The method for selecting records for review did not appear to affect event‐type representativeness, with similar distributions of event types and severities detected.
Conclusions
In this single‐site study of the EDTT we observed high levels of validity in trigger selection, yield, and representativeness of AEs, with yields that are superior to estimates for traditional approaches to AE detection. Record selection using weighted triggers outperforms a trigger count threshold approach and far outperforms random sampling from records with at least one trigger. The EDTT is a promising efficient and high‐yield approach for detecting all‐cause harm to guide quality improvement efforts in the ED.</abstract><cop>United States</cop><pub>Wiley Subscription Services, Inc</pub><pmid>32745284</pmid><doi>10.1111/acem.14101</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0002-6279-7380</orcidid><orcidid>https://orcid.org/0000-0001-7122-1079</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1069-6563 |
| ispartof | Academic emergency medicine, 2020-12, Vol.27 (12), p.1279-1290 |
| issn | 1069-6563 1553-2712 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_2430373720 |
| source | MEDLINE; Access via Wiley Online Library; EZB-FREE-00999 freely available EZB journals; Wiley Online Library (Open Access Collection) |
| subjects | Adolescent Adult Aged Electronic Health Records Emergency medical care Emergency Service, Hospital Humans Medical Errors - prevention & control Middle Aged Patient Safety Quality Improvement Retrospective Studies Side effects Vital signs Young Adult |
| title | The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-28T23%3A25%3A04IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Emergency%20Department%20Trigger%20Tool:%20Validation%20and%20Testing%20to%20Optimize%20Yield&rft.jtitle=Academic%20emergency%20medicine&rft.au=Griffey,%20Richard%20T.&rft.date=2020-12&rft.volume=27&rft.issue=12&rft.spage=1279&rft.epage=1290&rft.pages=1279-1290&rft.issn=1069-6563&rft.eissn=1553-2712&rft_id=info:doi/10.1111/acem.14101&rft_dat=%3Cproquest_cross%3E2471156798%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2471156798&rft_id=info:pmid/32745284&rfr_iscdi=true |