Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems
Background The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria. Methods We identified quality...
Gespeichert in:
| Veröffentlicht in: | Journal of general internal medicine : JGIM 2020-11, Vol.35 (Suppl 2), p.802-807 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 807 |
|---|---|
| container_issue | Suppl 2 |
| container_start_page | 802 |
| container_title | Journal of general internal medicine : JGIM |
| container_volume | 35 |
| creator | Hernandez, Adrian V. Roman, Yuani M. White, C. Michael |
| description | Background
The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria.
Methods
We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria.
Results
The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (
P
< 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (
P
< 0.001). However, the final revision was not significantly improved over the first revision (
P
= 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable).
Conclusion
Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ’s support in that regard. |
| doi_str_mv | 10.1007/s11606-020-06098-1 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7652974</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2471928973</sourcerecordid><originalsourceid>FETCH-LOGICAL-c474t-71111af71c27a6bd95ea0ed48b7bc3ae49573dc11ddc728f146e090869d0c09b3</originalsourceid><addsrcrecordid>eNp9kctuEzEUhi0EoqHwAiyQJTZsBnybsb1BqtJCI1VCqHRteWxP6mrGDrYnal6A58ZJSrks8MaWznc-n6MfgNcYvccI8Q8Z4w51DSKoQR2SosFPwAK3pG0wk_wpWCAhWCM4ZSfgRc53CGFKiHgOTigRSBDEF-DHudu6MW58WMNl8sUlr6EOFp7lHI3XxVm4Crmk2RQfQ4ZDTHBZHz4XF8oBvclumEf4ddajLzu4mjYpbt20L1-X2e7guS4aXtyXpA-Sg-PS6bHcwutd9Uz5JXg26DG7Vw_3Kbj5dPFtedlcffm8Wp5dNYZxVhqO69EDx4Zw3fVWtk4jZ5noeW-odky2nFqDsbWGEzFg1jkkkeikRQbJnp6Cj0fvZu4nZ02dMelRbZKfdNqpqL36uxL8rVrHreJdSyRnVfDuQZDi99nloiafjRtHHVycsyKMtljIju7Rt_-gd3FOoa5XKY4lEZLTSpEjZVLMObnhcRiM1D5mdYxZ1ZjVIWaFa9ObP9d4bPmVawXoEci1FNYu_f77P9qfo0m2UA</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2471928973</pqid></control><display><type>article</type><title>Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems</title><source>MEDLINE</source><source>Springer Nature - Complete Springer Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>PubMed Central</source><source>Alma/SFX Local Collection</source><creator>Hernandez, Adrian V. ; Roman, Yuani M. ; White, C. Michael</creator><creatorcontrib>Hernandez, Adrian V. ; Roman, Yuani M. ; White, C. Michael</creatorcontrib><description>Background
The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria.
Methods
We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria.
Results
The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (
P
< 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (
P
< 0.001). However, the final revision was not significantly improved over the first revision (
P
= 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable).
Conclusion
Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ’s support in that regard.</description><identifier>ISSN: 0884-8734</identifier><identifier>EISSN: 1525-1497</identifier><identifier>DOI: 10.1007/s11606-020-06098-1</identifier><identifier>PMID: 32808207</identifier><language>eng</language><publisher>Cham: Springer International Publishing</publisher><subject>Asthma ; Consistency ; Criteria ; Drug stores ; Evidence-based nursing ; Government Programs ; Humans ; Internal Medicine ; Intervention ; Medicine ; Medicine & Public Health ; Nurses ; Organizational change ; Original Research ; Pharmacy ; Quality assessment ; Quality control ; Quality Improvement ; Similarity ; United States ; United States Agency for Healthcare Research and Quality ; Working groups</subject><ispartof>Journal of general internal medicine : JGIM, 2020-11, Vol.35 (Suppl 2), p.802-807</ispartof><rights>Society of General Internal Medicine (This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply) 2020</rights><rights>Society of General Internal Medicine (This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply) 2020.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c474t-71111af71c27a6bd95ea0ed48b7bc3ae49573dc11ddc728f146e090869d0c09b3</citedby><cites>FETCH-LOGICAL-c474t-71111af71c27a6bd95ea0ed48b7bc3ae49573dc11ddc728f146e090869d0c09b3</cites><orcidid>0000-0002-9367-4893</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC7652974/pdf/$$EPDF$$P50$$Gpubmedcentral$$H</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC7652974/$$EHTML$$P50$$Gpubmedcentral$$H</linktohtml><link.rule.ids>230,314,723,776,780,881,27901,27902,41464,42533,51294,53766,53768</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/32808207$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Hernandez, Adrian V.</creatorcontrib><creatorcontrib>Roman, Yuani M.</creatorcontrib><creatorcontrib>White, C. Michael</creatorcontrib><title>Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems</title><title>Journal of general internal medicine : JGIM</title><addtitle>J GEN INTERN MED</addtitle><addtitle>J Gen Intern Med</addtitle><description>Background
The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria.
Methods
We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria.
Results
The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (
P
< 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (
P
< 0.001). However, the final revision was not significantly improved over the first revision (
P
= 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable).
Conclusion
Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ’s support in that regard.</description><subject>Asthma</subject><subject>Consistency</subject><subject>Criteria</subject><subject>Drug stores</subject><subject>Evidence-based nursing</subject><subject>Government Programs</subject><subject>Humans</subject><subject>Internal Medicine</subject><subject>Intervention</subject><subject>Medicine</subject><subject>Medicine & Public Health</subject><subject>Nurses</subject><subject>Organizational change</subject><subject>Original Research</subject><subject>Pharmacy</subject><subject>Quality assessment</subject><subject>Quality control</subject><subject>Quality Improvement</subject><subject>Similarity</subject><subject>United States</subject><subject>United States Agency for Healthcare Research and Quality</subject><subject>Working groups</subject><issn>0884-8734</issn><issn>1525-1497</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>8G5</sourceid><sourceid>BENPR</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp9kctuEzEUhi0EoqHwAiyQJTZsBnybsb1BqtJCI1VCqHRteWxP6mrGDrYnal6A58ZJSrks8MaWznc-n6MfgNcYvccI8Q8Z4w51DSKoQR2SosFPwAK3pG0wk_wpWCAhWCM4ZSfgRc53CGFKiHgOTigRSBDEF-DHudu6MW58WMNl8sUlr6EOFp7lHI3XxVm4Crmk2RQfQ4ZDTHBZHz4XF8oBvclumEf4ddajLzu4mjYpbt20L1-X2e7guS4aXtyXpA-Sg-PS6bHcwutd9Uz5JXg26DG7Vw_3Kbj5dPFtedlcffm8Wp5dNYZxVhqO69EDx4Zw3fVWtk4jZ5noeW-odky2nFqDsbWGEzFg1jkkkeikRQbJnp6Cj0fvZu4nZ02dMelRbZKfdNqpqL36uxL8rVrHreJdSyRnVfDuQZDi99nloiafjRtHHVycsyKMtljIju7Rt_-gd3FOoa5XKY4lEZLTSpEjZVLMObnhcRiM1D5mdYxZ1ZjVIWaFa9ObP9d4bPmVawXoEci1FNYu_f77P9qfo0m2UA</recordid><startdate>20201101</startdate><enddate>20201101</enddate><creator>Hernandez, Adrian V.</creator><creator>Roman, Yuani M.</creator><creator>White, C. Michael</creator><general>Springer International Publishing</general><general>Springer Nature B.V</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7QL</scope><scope>7RV</scope><scope>7U9</scope><scope>7X7</scope><scope>7XB</scope><scope>88C</scope><scope>8AO</scope><scope>8FD</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>C1K</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>H94</scope><scope>K9.</scope><scope>M0S</scope><scope>M0T</scope><scope>M1P</scope><scope>M2O</scope><scope>M7N</scope><scope>MBDVC</scope><scope>NAPCQ</scope><scope>P64</scope><scope>PHGZM</scope><scope>PHGZT</scope><scope>PJZUB</scope><scope>PKEHL</scope><scope>PPXIY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0002-9367-4893</orcidid></search><sort><creationdate>20201101</creationdate><title>Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems</title><author>Hernandez, Adrian V. ; Roman, Yuani M. ; White, C. Michael</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c474t-71111af71c27a6bd95ea0ed48b7bc3ae49573dc11ddc728f146e090869d0c09b3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Asthma</topic><topic>Consistency</topic><topic>Criteria</topic><topic>Drug stores</topic><topic>Evidence-based nursing</topic><topic>Government Programs</topic><topic>Humans</topic><topic>Internal Medicine</topic><topic>Intervention</topic><topic>Medicine</topic><topic>Medicine & Public Health</topic><topic>Nurses</topic><topic>Organizational change</topic><topic>Original Research</topic><topic>Pharmacy</topic><topic>Quality assessment</topic><topic>Quality control</topic><topic>Quality Improvement</topic><topic>Similarity</topic><topic>United States</topic><topic>United States Agency for Healthcare Research and Quality</topic><topic>Working groups</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Hernandez, Adrian V.</creatorcontrib><creatorcontrib>Roman, Yuani M.</creatorcontrib><creatorcontrib>White, C. Michael</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Bacteriology Abstracts (Microbiology B)</collection><collection>Nursing & Allied Health Database</collection><collection>Virology and AIDS Abstracts</collection><collection>Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Healthcare Administration Database (Alumni)</collection><collection>ProQuest Pharma Collection</collection><collection>Technology Research Database</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Environmental Sciences and Pollution Management</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>Healthcare Administration Database</collection><collection>Medical Database</collection><collection>Research Library</collection><collection>Algology Mycology and Protozoology Abstracts (Microbiology C)</collection><collection>Research Library (Corporate)</collection><collection>Nursing & Allied Health Premium</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>ProQuest Central (New)</collection><collection>ProQuest One Academic (New)</collection><collection>ProQuest Health & Medical Research Collection</collection><collection>ProQuest One Academic Middle East (New)</collection><collection>ProQuest One Health & Nursing</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Journal of general internal medicine : JGIM</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Hernandez, Adrian V.</au><au>Roman, Yuani M.</au><au>White, C. Michael</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems</atitle><jtitle>Journal of general internal medicine : JGIM</jtitle><stitle>J GEN INTERN MED</stitle><addtitle>J Gen Intern Med</addtitle><date>2020-11-01</date><risdate>2020</risdate><volume>35</volume><issue>Suppl 2</issue><spage>802</spage><epage>807</epage><pages>802-807</pages><issn>0884-8734</issn><eissn>1525-1497</eissn><abstract>Background
The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria.
Methods
We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria.
Results
The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (
P
< 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (
P
< 0.001). However, the final revision was not significantly improved over the first revision (
P
= 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable).
Conclusion
Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ’s support in that regard.</abstract><cop>Cham</cop><pub>Springer International Publishing</pub><pmid>32808207</pmid><doi>10.1007/s11606-020-06098-1</doi><tpages>6</tpages><orcidid>https://orcid.org/0000-0002-9367-4893</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0884-8734 |
| ispartof | Journal of general internal medicine : JGIM, 2020-11, Vol.35 (Suppl 2), p.802-807 |
| issn | 0884-8734 1525-1497 |
| language | eng |
| recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7652974 |
| source | MEDLINE; Springer Nature - Complete Springer Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; PubMed Central; Alma/SFX Local Collection |
| subjects | Asthma Consistency Criteria Drug stores Evidence-based nursing Government Programs Humans Internal Medicine Intervention Medicine Medicine & Public Health Nurses Organizational change Original Research Pharmacy Quality assessment Quality control Quality Improvement Similarity United States United States Agency for Healthcare Research and Quality Working groups |
| title | Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-20T20%3A26%3A20IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Developing%20Criteria%20and%20Associated%20Instructions%20for%20Consistent%20and%20Useful%20Quality%20Improvement%20Study%20Data%20Extraction%20for%20Health%20Systems&rft.jtitle=Journal%20of%20general%20internal%20medicine%20:%20JGIM&rft.au=Hernandez,%20Adrian%20V.&rft.date=2020-11-01&rft.volume=35&rft.issue=Suppl%202&rft.spage=802&rft.epage=807&rft.pages=802-807&rft.issn=0884-8734&rft.eissn=1525-1497&rft_id=info:doi/10.1007/s11606-020-06098-1&rft_dat=%3Cproquest_pubme%3E2471928973%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2471928973&rft_id=info:pmid/32808207&rfr_iscdi=true |