Similarity-based sampling for simulation with binary outcomes

We consider a feasibility determination problem via simulation with stochastic binary outcomes, in which the design space can be either discrete or continuous, and outcomes can be predicted through a functional relationship that depends on linear combinations of design variables. The goal is to identify all the feasible designs with means (i.e., probabilities) no smaller than a threshold. A logistic model is used to capture the relationship between the probability and design variables. Traditional binary rewards often conceal the numbers of correct and false determinations, thereby being inefficient in large and continuous design spaces. We thus propose a similarity measure to smooth binary rewards. Then, a sampling policy that optimizes a so-called Similarity Differential (SD) is developed. Under some mild conditions, we show that the SD policy is capable of identifying all the feasible designs as the sampling budget goes to infinity. Two approximate versions of the SD policy are developed to sequentially determine the sampling decisions in large and continuous design spaces. Extensive numerical experiments are conducted to demonstrate the superior performance of our SD policy, document computational savings, and reveal underlying sampling behaviors. Alternatively, we provide a simple but effective heuristic that can be easily used by practitioners.