Problem with p values: why p values do not tell you if your treatment is likely to work

Worse still, as these ideas became broadly adopted, fundamental misinterpretations were embedded in the literature and practice of biomedical research.1 Fisher originally proposed using the exact p value for a single trial as an indication of the credibility of the null hypothesis when considered together with all the available evidence.2 It is worth noting that the null hypothesis does not necessarily mean no difference between groups, this is the nil null hypothesis.3 Rather it is the hypothesis we aim to nullify by experiment, which can provide more powerful evidence if it includes a quantitative prediction of the expected difference. Despite the inherent conflict in these two approaches, they have been fused into NHST for single trials, where a p value threshold is used to accept or reject the null hypothesis.2 The problem with p values and their misinterpretation For several decades, there has been a failure by many authors to realise that all these probabilities (p, α, β, type I error, type II error) are conditional.4 The order of terms matters in conditional probabilities, and they must be used according to a simple set of mathematical rules or their meanings are changed. First of all, we need to teach the correct statistical interpretation of NHST because of the huge volume of trials already published. [...]we need to move to statistical models that are better suited to current research problems and address some of the shortcomings of NHST.