Assessments of Conditional and Unconditional Type I Error Probabilities for Bayesian Hypothesis Testing with Historical Data Borrowing

Applications of Bayesian designs allow the borrowing of the strength of historical information and become more and more attractive in new drug developments. Nonetheless, according to the FDA guidance issued in 2020, Bayesian designs are classified as complex innovative designs that have rarely been used to provide substantial evidence of effectiveness in new drug applications. Moreover, as the historical data have already been observed and fixed, a question which arises is whether we should treat the Bayesian analysis as a conditional or unconditional analysis. Basically, it is essential to understand the frequentist operating characteristics of a Bayesian design either theoretically or through simulation in order to appropriately assess the right type I error probability and apply it to a clinical trial. In this research, we use a relatively simple setting of a normal distribution for the study endpoint to illustrate and compare the conditional and unconditional Bayesian analysis. Both scenarios of borrowing historical information of treatment effect and historical control data are considered. The thinking is applicable to the other settings or endpoints through the asymptotic normality of the distributions for the estimators of either the within or between treatment effects. Simulations are conducted to evaluate the characteristics of the methods.