Methodological Quality of Surgical Mortality Studies Using Large Hospital Databases: A Systematic Review

To review the methodology employed in surgical mortality studies to control for potential confounders. Nationwide hospital data are increasingly used to investigate surgical outcomes. However, poor data granularity and coding inaccuracies may lead to flawed findings. We conducted a systematic review in accordance with the PRISMA statement in 6 major journals (NEJM, Lancet, BMJ, JAMA, Medical Care, Annals of Surgery) using PubMed from its inception until December 31, 2014. Two reviewers independently reviewed citations. Using a predesigned data collection form, we extracted information about study aim and design, data source, selected population, outcome definition, patient and hospital adjustment, statistics, and sensitivity analyses. The methodological quality of studies was assessed based on 5 criteria and explored over time. Among 89 included studies from 1987 to 2014, 54 explored surgical mortality determinants, 13 compared surgical procedure effectiveness, 13 evaluated the impact of healthcare policy, and 9 described outcome trends for specific procedures. A total of 89% (n = 79) of studies did not describe population selection criteria at patient and hospital level, 64% (n = 57) did not consider secular trends, 52% (n = 46) neglected hospital clustering or characteristics, 21% (n = 19) did not perform sensitivity analyses, and 4% did not adjust outcomes for patient risk (n = 4). The percentage of studies satisfying at least 3 of these criteria increased significantly from 44% before 1999 to 52% between 2000 and 2009 and 78% after 2010 (P = 0.008). Although methodological quality of studies has improved over time, confounder control could be improved through better study design, homogeneous population selection, the consideration of hospital factors and secular trends influencing surgical mortality, and the systematic performance of sensitivity analyses.