Comparison of MC4PC and MDL-QSAR rodent carcinogenicity predictions and the enhancement of predictive performance by combining QSAR models

This report presents a comparison of the predictive performance of MC4PC and MDL-QSAR software as well as a method for combining the predictions from both programs to increase overall accuracy. The conclusions are based on 10 × 10% leave-many-out internal cross-validation studies using 1540 training set compounds with 2-year rodent carcinogenicity findings. The models were generated using the same weight of evidence scoring method previously developed [Matthews, E.J., Contrera, J.F., 1998. A new highly specific method for predicting the carcinogenic potential of pharmaceuticals in rodents using enhanced MCASE QSAR-ES software. Regul. Toxicol. Pharmacol. 28, 242–264.]. Although MC4PC and MDL-QSAR use different algorithms, their overall predictive performance was remarkably similar. Respectively, the sensitivity of MC4PC and MDL-QSAR was 61 and 63%, specificity was 71 and 75%, and concordance was 66 and 69%. Coverage for both programs was over 95% and receiver operator characteristic (ROC) intercept statistic values were above 2.00. The software programs had complimentary coverage with none of the 1540 compounds being uncovered by both MC4PC and MDL-QSAR. Merging MC4PC and MDL-QSAR predictions improved the overall predictive performance. Consensus sensitivity increased to 67%, specificity to 84%, concordance to 76%, and ROC to 4.31. Consensus rules can be tuned to reflect the priorities of the user, so that greater emphasis may be placed on predictions with high sensitivity/low false negative rates or high specificity/low false positive rates. Sensitivity was optimized to 75% by reclassifying all compounds predicted to be positive in MC4PC or MDL-QSAR as positive, and specificity was optimized to 89% by reclassifying all compounds predicted negative in MC4PC or MDL-QSAR as negative.