Identification of practitioners at high risk of complaints to health profession regulators

Some health practitioners pose substantial threats to patient safety, yet early identification of them is notoriously difficult. We aimed to develop an algorithm for use by regulators in prospectively identifying practitioners at high risk of attracting formal complaints about health, conduct or performance issues. Using 2011-2016 data from the national regulator of health practitioners in Australia, we conducted a retrospective cohort study of 14 registered health professions. We used recurrent-event survival analysis to estimate the risk of a complaint and used the results of this analysis to develop an algorithm for identifying practitioners at high risk of complaints. We evaluated the algorithm's discrimination, calibration and predictive properties. Participants were 715,415 registered health practitioners (55% nurses, 15% doctors, 6% midwives, 5% psychologists, 4% pharmacists, 15% other). The algorithm, PRONE-HP (Predicted Risk of New Event for Health Practitioners), incorporated predictors for sex, age, profession and specialty, number of prior complaints and complaint issue. Discrimination was good (C-index = 0·77, 95% CI 0·76-0·77). PRONE-HP's score values were closely calibrated with risk of a future complaint: practitioners with a score ≤ 4 had a 1% chance of a complaint within 24 months and those with a score ≥ 35 had a higher than 85% chance. Using the 90th percentile of scores within each profession to define "high risk", the predictive accuracy of PRONE-HP was good for doctors and dentists (PPV = 93·1% and 91·6%, respectively); moderate for chiropractors (PPV = 71·1%), psychologists (PPV = 54·9%), pharmacists (PPV = 39·9%) and podiatrists (PPV = 34·0%); and poor for other professions. The performance of PRONE-HP in predicting complaint risks varied substantially across professions. It showed particular promise for flagging doctors and dentists at high risk of accruing further complaints. Close review of available information on flagged practitioners may help to identify troubling patterns and imminent risks to patients.