Waldian t Tests: Sequential Bayesian t Tests With Controlled Error Probabilities

Bayesian t tests have become increasingly popular alternatives to null-hypothesis significance testing (NHST) in psychological research. In contrast to NHST, they allow for the quantification of evidence in favor of the null hypothesis and for optional stopping. A major drawback of Bayesian t tests, however, is that error probabilities of statistical decisions remain uncontrolled. Previous approaches in the literature to remedy this problem require time-consuming simulations to calibrate decision thresholds. In this article, we propose a sequential probability ratio test that combines Bayesian t tests with simple decision criteria developed by Abraham Wald in 1947. We discuss this sequential procedure, which we call Waldian t test, in the context of three recently proposed specifications of Bayesian t tests. Waldian t tests preserve the key idea of Bayesian t tests by assuming a distribution for the effect size under the alternative hypothesis. At the same time, they control expected frequentist error probabilities, with the nominal Type I and Type II error probabilities serving as upper bounds to the actual expected error rates under the specified statistical models. Thus, Waldian t tests are fully justified from both a Bayesian and a frequentist point of view. We highlight the relationship between Bayesian and frequentist error probabilities and critically discuss the implications of conventional stopping criteria for sequential Bayesian t tests. Finally, we provide a user-friendly web application that implements the proposed procedure for interested researchers. Translational Abstract Bayesian t tests have become increasingly popular in psychological research. In contrast to classical test procedures, Bayesian tests can measure statistical evidence in favor of the null hypothesis and allow for optional stopping. Yet, probabilities of statistical decision errors (i.e., falsely rejecting a hypothesis when it is true) are not explicitly controlled. In this article, we propose a sequential test procedure where Bayesian t tests are calculated repeatedly after each additional observation. The sample size is increased until the test exceeds a predefined threshold. We call the proposed procedure Waldian t test because it is a straightforward combination of Bayesian t tests with Abraham Wald's sequential probability ratio test. We illustrate the procedure in the context of three different types of default and informed Bayesian t tests, and show how it satisfies