Patient Safety Indicators are an insufficient performance metric to track and grade outcomes of open aortic repair

National rankings of hospitals rely on outcomes-based evaluation to assess the performance of surgical programs, particularly those performing high-risk elective surgical procedures such as open aortic repair. Various classification systems exist for tracking outcomes, but increasingly the International Classification of Diseases, Tenth Revision-based Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) are used as a publicly reported comparison measure of hospital quality performance. We sought to critically evaluate the accuracy of the existing vehicles to assess open aortic repair outcomes in an established program. This is a case-control study of patients who underwent open abdominal aortic aneurysm repair at the Johns Hopkins Medical Institutions from 2004 to 2018. Patients' characteristics and outcomes were collected as part of a prospectively maintained retrospective database. For each case, hemorrhagic, cardiac, respiratory, renal, wound, and thromboembolic complications were identified with the unique definitions used for open abdominal aortic aneurysm repair by the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) database, the Society for Vascular Surgery Vascular Quality Initiative (VQI) database, and the Agency for Healthcare Research and Quality PSI initiative. Of the 154 patients included in the study, 79 (51.0%) were identified as having a complication as defined by the VQI, 46 (29.7%) according to the NSQIP, and 15 (9.7%) according to the PSI system (P < .001). Patients most likely to incur a complication in the PSI system were those with a pararenal or more extensive aneurysm, with baseline congestive heart failure, requiring a supramesenteric clamp (all P < .01), or with an aneurysm >6.5 cm in diameter (P = .02). The NSQIP and VQI systems both identified more postoperative hemorrhagic, respiratory, renal, and wound complications than the PSI system did (P < .05). The VQI system identified the most renal complications (52; P < .001); factors unique to incurring a complication in the VQI include use of a suprarenal clamp and performance of an aortorenal bypass procedure as part of the repair (P < .01). Particularly weak correlation was noted between the PSI system and the VQI with respect to renal outcomes (ρ = 0.163). The PSI system identified fewer important complications than either of the clinically focused databases, with the VQI capturing the most postoperative events, mostly be