Advancing statistical decision-making in sports science

The magnitude-based decisions (MBD) procedure was developed within sports science as an alternative to null hypothesis significance tests. It aimed to emphasise effect sizes and discourage dichotomous decision-making. The use of MBD was banned by some sports science journals following claims it lacks a theoretical foundation and leads to high Type I error rates. To address these claims, we first generalise contour-enhanced funnel plots to allow for ranges of meaningful effect sizes, then relate regions defined in these plots to the decisions made by MBD. We then mathematically show how MBD fits within a class of multiple decision procedures. We have implemented this theoretically sound version of MBD as a visualisation tool that supports generalised funnel plots. The use of MBD could encourage researchers to plan test directionalities, test levels and error definitions, and the visualisation tool may help stakeholders engage with the design of analyses and the interpretation of trial findings.