Poor concordance of contemporary performance measures in detecting complications in complex endovascular aortic repair

The Agency for Healthcare Research and Quality Patient Safety Indicators (PSI) are quality improvement indicators used to determine hospital performance and, increasingly, to rank surgical programs. The American College of Surgeons National Surgical Quality Improvement Program and the Society for Vascular Surgery Vascular Quality Improvement databases are also frequently used to compare outcomes, but definitions of complications vary between the systems and the optimal system for tracking complications in complex endovascular repair remains unclear. Herein we assess the three outcome tracking systems and their ability to capture complications after fenestrated endovascular abdominal aortic aneurysm repair (FEVAR) and open aortic aneurysm repair in a large complex aortic program. Demographic and operative data for patients undergoing repair of juxtarenal or pararenal aortic aneurysms between 2004 and 2018 via both open and FEVAR approaches at the Johns Hopkins Medical Institutions were compiled in a prospectively maintained retrospective database. Postoperative complications were defined according to a surgeon-defined system, the Society for Vascular Surgery Vascular Quality Initiative, the American College of Surgeons National Surgical Quality Improvement Program, and the Agency for Healthcare Research and Quality PSI data dictionaries and were compared between surgical approaches as well as eras before and after the introduction of FEVAR. Complication rates between the classification systems were compared using proportion testing and the strength of the correlation between the systems was evaluated with Spearman's rank test. Of 145 patients, 60 (41.4%) underwent FEVAR and 85 (58.6%) underwent open aortic aneurysm repair. The introduction of fenestrated technology was associated with a decrease in the overall number of complications from 37.2% to 20.6% by surgeon-defined classification system (P = .036). The VQI identified the most complications (39.9% of the entire cohort and 25% of FEVAR cases), followed by the NSQIP (29.0% and 33.3%, respectively) and PSI (4.1% and 5%). The two clinically focused databases were found to correlate well with a surgeon-designed classification system, as well as each other (Spearman ρ ≥ 0.735) but not with PSI (ρ < 0.23). Proportion testing demonstrated the rate of complications identified by PSI to be significantly less than either VQI or NSQIP (P < .001). Specifically, PSI did not effectively identify renal complications