Between therapy effect and false-positive result in animal experimentation

Despite the animal models’ complexity, researchers tend to reduce the number of animals in experiments for expenses and ethical concerns. This tendency makes the risk of false-positive results, as statistical significance, the primary criterion to validate findings, often fails if testing small samples. This study aims to highlight such risks using an example from experimental regenerative therapy and propose a machine-learning solution to validate treatment effects. The example analysed was the pharmacological treatment of ear pinna punch wound healing in mice. Wound closure data analysed included eight groups treated with an epigenetic inhibitor, zebularine, and eight control groups receiving vehicle alone, of six mice each. We confirmed the zebularine healing effect for all 64 pairwise comparisons between treatment and control groups but also determined minor yet statistically significant differences between control groups in five of 28 possible comparisons. The occurrences of significant differences between the control groups, regardless of standardised experimental conditions, indicate a risk of statistically significant effects in the case a compound lacking the desired biological activity is tested. Since the criterion of statistical significance itself can be confusing, we demonstrate a machine-learning algorithm trained on datasets representing treatment and control experiments as a helpful tool for validating treatment outcomes. We tested two machine-learning approaches, Naïve Bayes and Support Vector Machine classifiers. In contrast to the Mann-Whitney U-test, indicating enhanced healing effects for some control groups receiving saline alone, both machine-learning algorithms faultlessly assigned all animal groups receiving saline to the controls. [Display omitted] •Researchers tend to reduce the number of animals used in experiments.•Small sample sizes increase the risk of false-positive findings.•Significant p-values may result from technical and biological variation.•Machine learning classifiers proved effective in identifying false-positives.