Thresholds and innovation

[...]too often authors do not recognize that ‘no statistical significance’ is not the same as ‘no effect’ and that ‘statistically significant’ does not mean ‘proven’. [...]they often ignore the essential difference between statistical significance and clinical relevance of an effect (that must be a priori elaborated as minimal important effect to be detected, or as threshold for clinical importance [5]) As a consequence, clinicians, patients, media, and policy makers may be misled, and harm may be the result. Bayesian analysis provides a much more straightforward interpretation than frequentist analysis, but its estimation of the (posterior) probability of a hypothesis being true also starts from a dichotomous hypothesis, and thresholds for posterior probabilities are often used. [...]Bayesian approaches are confronted with challenges such as prior probabilities being unknown, highly dependent of specific situations, or quite subjective [7]. Where thresholds are used to support decision making, one should make sure that this is well justified, and based on an appropriately predefined effect and sample size. The authors recommend that these tests should be superseded by methods that are not dependent on the number of studies. Since validity of study designs and appropriate use of all relevant evidence are a prerequisite for any statistical analysis, evaluating the methodological performance of the scientific community – always striving for innovation - is an important mission of our journal.