Reconsideration of the Type I Error Rate for Psychological Science in the Era of Replication

Careful consideration of the tradeoff between Type I and Type II error rates when designing experiments is critical for maximizing statistical decision accuracy. Typically, Type I error rates (e.g., .05) are significantly lower than Type II error rates (e.g., .20 for .80 power) in psychological science. Further, positive findings (true effects and Type I errors) are more likely to be the focus of replication. This conventional approach leads to very high rates of Type II error. Analyses show that increasing the Type I error rate to .10, thereby increasing power and decreasing the Type II error rate for each test, leads to higher overall rates of correct statistical decisions. This increase of Type I error rate is consistent with, and most beneficial in the context of, the replication and "New Statistics" movements in psychology. Translational Abstract Making decisions under conditions of uncertainty (e.g., science) always involves a tradeoff between two types of errors. The decision maker can conclude something is present when it is not (false positive or Type I error) or conclude no difference is present, when in reality there is a difference (a miss or Type II error). We address the need to carefully consider the balance of these error types in scientific decision making. In psychology, it is typical to more stringently control the Type I error rate at .05, while the Type II error rate is .20 (i.e., when power is .80). We demonstrate that the overall percentage correct decisions in psychological science can be increased by increasing the Type I error rate to .10 to increase power, and decrease the Type II error rate. This is particularly relevant given the recent methodological and statistical changes in the field, particularly the long-needed emphasis on replication.