Proposal of a Standardized Questionnaire to Structure Clinical Peer Reviews of Mortality and Failure of Rescue in Pancreatic Surgery

Background: Quality management tools such as clinical peer reviews facilitate root cause analysis and may, ultimately, help to reduce surgery-related morbidity and mortality. This study aimed to evaluate the reliability of a standardized questionnaire for clinical peer reviews in pancreatic surgery. Methods: All cases of in-hospital-mortality following pancreatic surgery at two high-volume centers (n = 86) were reviewed by two pancreatic surgeons. A standardized mortality review questionnaire was developed and applied to all cases. In a second step, 20 cases were randomly assigned to an online re-review that was completed by seven pancreatic surgeons. The overall consistency of the results between the peer review and online re-review was determined by Cohen's kappa (kappa). The inter-rater reliability of the online re-review was assessed by Fleiss' kappa (kappa). Results: The clinical peer review showed that 80% of the patient mortality was related to surgery. Post-operative pancreatic fistula (POPF) (36%) followed by post-pancreatectomy hemorrhage (PPH) (22%) were the most common surgical underlying (index) complications leading to in-hospital mortality. Most of the index complications yielded in abdominal sepsis (62%); 60% of the cases exhibited potential of improvement, especially through timely diagnosis and therapy (42%). There was a moderate to substantial strength of agreement between the peer review and the online re-review in regard to the category of death (surgical vs. non-surgical; kappa = 0.886), type of surgical index complication (kappa = 0.714) as well as surgical and non-surgical index complications (kappa = 0.492 and kappa = 0.793). Fleiss' kappa showed a moderate to substantial inter-rater agreement of the online re-review in terms of category of death (kappa = 0.724), category of common surgical index complications (kappa = 0.455) and surgical index complication (kappa = 0.424). Conclusion: The proposed questionnaire to structure clinical peer reviews is a reliable tool for root cause analyses of in-hospital mortality and may help to identify specific options to improve outcomes in pancreatic surgery. However, the reliability of the peer feedback decreases with an increasing specificity of the review questions.