Covariates adjustment questioned conclusions of predictive analyses: an illustration with the Kidney Donor Risk Index

We aimed to illustrate that considering covariates can lead to meaningful interpretation of the discriminative capacities of a prognostic marker. For this, we evaluated the ability of the Kidney Donor Risk Index (KDRI) to discriminate kidney graft failure risk. From 4114 French patients, we estimated the adjusted area under the time-dependent ROC curve by standardizing the marker and weighting the observations. By weighting the contributions, we also studied the impact of KDRI-based transplantations on the patient and graft survival. The covariate-adjusted AUC varied from 55% (95% confidence interval [CI]: 51–60%) for a prognostic up to 1 year post-transplantation to 56% (95% CI: 52–59%) up to 7 years. The Restricted Mean Survival Time (RMST) was 6.44 years for high-quality graft recipients (95% CI: 6.30–6.56) and would have been 6.31 years (95% CI: 6.13–6.46) if they had medium-quality transplants. The RMST was 5.10 years for low-quality graft recipients (95% CI: 4.90–5.31) and would have been 5.52 years (95% CI: 5.17–5.83) if they had medium-quality transplants. We demonstrated that the KDRI discriminative capacities were mainly explained by the recipient characteristics. We also showed that counterfactual estimations, often used in causal studies, are also interesting in predictive studies, especially regarding the new available methods.