Clinical decisions support malfunctions in a commercial electronic health record
Summary Objectives : Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection. Methods : We retrospectively examined the firing rate for 226 alert type CDS rules for detection...
Gespeichert in:
| Veröffentlicht in: | Applied clinical informatics 2017-09, Vol.8 (3), p.910-923 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 923 |
|---|---|
| container_issue | 3 |
| container_start_page | 910 |
| container_title | Applied clinical informatics |
| container_volume | 8 |
| creator | Kassakian, Steven Z. Yackel, Thomas R. Gorman, Paul N. Dorr, David A. |
| description | Summary
Objectives
: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.
Methods
: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.
Results
: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.
Discussion
: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.
Conclusion
: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.
Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910–923 https://doi.org/10.4338/ACI-2017-01-RA-0006 |
| doi_str_mv | 10.4338/ACI-2017-01-RA-0006 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6220702</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1936622014</sourcerecordid><originalsourceid>FETCH-LOGICAL-c448t-abe43016686621fbce04baf7b8a6de0f47d542912201b9f409701ef8e2cb26bd3</originalsourceid><addsrcrecordid>eNp9kU9LxDAQxYMo7qJ-AkF69FKdpDFNL8Ky-GdBUETPIU2nbiRt1qQV_PZmXZX14ilD8ntvMvMIOaZwxotCns_mi5wBLXOg-eMsBwCxQ6ZUiiqHgpW7W_WEHMX4mgi4EFTKcp9MmJQSgIspeZg721ujXdagsdH6PmZxXK18GLJOu3bszfB1aftMZ8Z3HQZjE44OzRB80mZL1G5YZgGND80h2Wu1i3j0fR6Q5-urp_ltfnd_s5jP7nLDuRxyXSMvgAohhWC0rQ0Cr3Vb1lKLBqHlZXPBWUVZGrKuWg5VCRRbiczUTNRNcUAuN76rse6wMdgPQTu1CrbT4UN5bdXfl94u1Yt_VyJZlsCSwem3QfBvI8ZBdTYadE736MeoaFWINUt5QosNaoKPMWD724aCWsehUhxqHYcCqh5nah1HUp1s__BX87P8BLANMCwtdqhe_Rj6tLN_XT8B6VeXfA</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1936622014</pqid></control><display><type>article</type><title>Clinical decisions support malfunctions in a commercial electronic health record</title><source>MEDLINE</source><source>EZB-FREE-00999 freely available EZB journals</source><source>PubMed Central</source><creator>Kassakian, Steven Z. ; Yackel, Thomas R. ; Gorman, Paul N. ; Dorr, David A.</creator><creatorcontrib>Kassakian, Steven Z. ; Yackel, Thomas R. ; Gorman, Paul N. ; Dorr, David A.</creatorcontrib><description>Summary
Objectives
: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.
Methods
: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.
Results
: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.
Discussion
: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.
Conclusion
: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.
Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910–923 https://doi.org/10.4338/ACI-2017-01-RA-0006</description><identifier>ISSN: 1869-0327</identifier><identifier>EISSN: 1869-0327</identifier><identifier>DOI: 10.4338/ACI-2017-01-RA-0006</identifier><identifier>PMID: 28880046</identifier><language>eng</language><publisher>Germany: Schattauer GmbH</publisher><subject>Alert Fatigue, Health Personnel ; Coronary Artery Disease - drug therapy ; Decision Support Systems, Clinical ; Documentation ; Electronic Health Records ; False Positive Reactions ; Humans ; Influenza Vaccines - administration & dosage ; Medical Errors ; Medical Order Entry Systems ; Neoplasms ; Research Article</subject><ispartof>Applied clinical informatics, 2017-09, Vol.8 (3), p.910-923</ispartof><rights>Thieme Medical Publishers</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c448t-abe43016686621fbce04baf7b8a6de0f47d542912201b9f409701ef8e2cb26bd3</citedby><cites>FETCH-LOGICAL-c448t-abe43016686621fbce04baf7b8a6de0f47d542912201b9f409701ef8e2cb26bd3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6220702/pdf/$$EPDF$$P50$$Gpubmedcentral$$H</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6220702/$$EHTML$$P50$$Gpubmedcentral$$H</linktohtml><link.rule.ids>230,314,723,776,780,881,27901,27902,53766,53768</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/28880046$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Kassakian, Steven Z.</creatorcontrib><creatorcontrib>Yackel, Thomas R.</creatorcontrib><creatorcontrib>Gorman, Paul N.</creatorcontrib><creatorcontrib>Dorr, David A.</creatorcontrib><title>Clinical decisions support malfunctions in a commercial electronic health record</title><title>Applied clinical informatics</title><addtitle>Appl Clin Inform</addtitle><description>Summary
Objectives
: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.
Methods
: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.
Results
: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.
Discussion
: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.
Conclusion
: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.
Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910–923 https://doi.org/10.4338/ACI-2017-01-RA-0006</description><subject>Alert Fatigue, Health Personnel</subject><subject>Coronary Artery Disease - drug therapy</subject><subject>Decision Support Systems, Clinical</subject><subject>Documentation</subject><subject>Electronic Health Records</subject><subject>False Positive Reactions</subject><subject>Humans</subject><subject>Influenza Vaccines - administration & dosage</subject><subject>Medical Errors</subject><subject>Medical Order Entry Systems</subject><subject>Neoplasms</subject><subject>Research Article</subject><issn>1869-0327</issn><issn>1869-0327</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp9kU9LxDAQxYMo7qJ-AkF69FKdpDFNL8Ky-GdBUETPIU2nbiRt1qQV_PZmXZX14ilD8ntvMvMIOaZwxotCns_mi5wBLXOg-eMsBwCxQ6ZUiiqHgpW7W_WEHMX4mgi4EFTKcp9MmJQSgIspeZg721ujXdagsdH6PmZxXK18GLJOu3bszfB1aftMZ8Z3HQZjE44OzRB80mZL1G5YZgGND80h2Wu1i3j0fR6Q5-urp_ltfnd_s5jP7nLDuRxyXSMvgAohhWC0rQ0Cr3Vb1lKLBqHlZXPBWUVZGrKuWg5VCRRbiczUTNRNcUAuN76rse6wMdgPQTu1CrbT4UN5bdXfl94u1Yt_VyJZlsCSwem3QfBvI8ZBdTYadE736MeoaFWINUt5QosNaoKPMWD724aCWsehUhxqHYcCqh5nah1HUp1s__BX87P8BLANMCwtdqhe_Rj6tLN_XT8B6VeXfA</recordid><startdate>20170906</startdate><enddate>20170906</enddate><creator>Kassakian, Steven Z.</creator><creator>Yackel, Thomas R.</creator><creator>Gorman, Paul N.</creator><creator>Dorr, David A.</creator><general>Schattauer GmbH</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>5PM</scope></search><sort><creationdate>20170906</creationdate><title>Clinical decisions support malfunctions in a commercial electronic health record</title><author>Kassakian, Steven Z. ; Yackel, Thomas R. ; Gorman, Paul N. ; Dorr, David A.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c448t-abe43016686621fbce04baf7b8a6de0f47d542912201b9f409701ef8e2cb26bd3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Alert Fatigue, Health Personnel</topic><topic>Coronary Artery Disease - drug therapy</topic><topic>Decision Support Systems, Clinical</topic><topic>Documentation</topic><topic>Electronic Health Records</topic><topic>False Positive Reactions</topic><topic>Humans</topic><topic>Influenza Vaccines - administration & dosage</topic><topic>Medical Errors</topic><topic>Medical Order Entry Systems</topic><topic>Neoplasms</topic><topic>Research Article</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kassakian, Steven Z.</creatorcontrib><creatorcontrib>Yackel, Thomas R.</creatorcontrib><creatorcontrib>Gorman, Paul N.</creatorcontrib><creatorcontrib>Dorr, David A.</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Applied clinical informatics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kassakian, Steven Z.</au><au>Yackel, Thomas R.</au><au>Gorman, Paul N.</au><au>Dorr, David A.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Clinical decisions support malfunctions in a commercial electronic health record</atitle><jtitle>Applied clinical informatics</jtitle><addtitle>Appl Clin Inform</addtitle><date>2017-09-06</date><risdate>2017</risdate><volume>8</volume><issue>3</issue><spage>910</spage><epage>923</epage><pages>910-923</pages><issn>1869-0327</issn><eissn>1869-0327</eissn><abstract>Summary
Objectives
: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.
Methods
: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.
Results
: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.
Discussion
: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.
Conclusion
: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.
Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910–923 https://doi.org/10.4338/ACI-2017-01-RA-0006</abstract><cop>Germany</cop><pub>Schattauer GmbH</pub><pmid>28880046</pmid><doi>10.4338/ACI-2017-01-RA-0006</doi><tpages>14</tpages><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1869-0327 |
| ispartof | Applied clinical informatics, 2017-09, Vol.8 (3), p.910-923 |
| issn | 1869-0327 1869-0327 |
| language | eng |
| recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6220702 |
| source | MEDLINE; EZB-FREE-00999 freely available EZB journals; PubMed Central |
| subjects | Alert Fatigue, Health Personnel Coronary Artery Disease - drug therapy Decision Support Systems, Clinical Documentation Electronic Health Records False Positive Reactions Humans Influenza Vaccines - administration & dosage Medical Errors Medical Order Entry Systems Neoplasms Research Article |
| title | Clinical decisions support malfunctions in a commercial electronic health record |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-21T18%3A52%3A48IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Clinical%20decisions%20support%20malfunctions%20in%20a%20commercial%20electronic%20health%20record&rft.jtitle=Applied%20clinical%20informatics&rft.au=Kassakian,%20Steven%20Z.&rft.date=2017-09-06&rft.volume=8&rft.issue=3&rft.spage=910&rft.epage=923&rft.pages=910-923&rft.issn=1869-0327&rft.eissn=1869-0327&rft_id=info:doi/10.4338/ACI-2017-01-RA-0006&rft_dat=%3Cproquest_pubme%3E1936622014%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1936622014&rft_id=info:pmid/28880046&rfr_iscdi=true |