Model‐based Bayesian inference under computer assisted balance‐improving designs

To improve covariate balance over a complete randomization, a number of methods have been proposed recently to utilize modern computational capabilities to find allocations with balance in observed covariates. Asymptotic inference on treatment effects based on these designs is more complicated than that under complete randomization, and this is why Fisher randomization tests often are suggested. This article suggests model‐based Bayesian inference as a general method of inference in these designs, which can deal with complications such as arbitrary covariate balancing criteria and complex estimands. As an illustration, we focus on the case when the outcome is linearly related to the covariates and the estimand of interest is the Sample Average Treatment Effect (SATE). We use a large Monte Carlo simulation to compare the finite sample performance of the model‐based Bayesian inference with that of two previous methods which are valid for asymptotic inference of SATE under Mahalanobis distance based rerandomization. We find that for experiments with small to moderate sample sizes, Bayesian inference is to be preferred to the previous methods. As a byproduct, we also find that regression adjustment together with small sample adjusted estimators of standard errors perform better than the previous methods.