Finding probably best systems quickly via simulations

We propose an indifference-zone approach for a ranking and selection problem with the goal of reducing both the number of simulated samples of the performance and the frequency of configuration changes. We prove that with a prespecified high probability, our algorithm finds the best system configuration. Our proof hinges on several ideas, including the use of Anderson's probability bound, that have not been fully investigated for the ranking and selection problem. Numerical experiments show that our algorithm can select the best system configuration using up to 50% fewer simulated samples than existing algorithms without increasing the frequency of configuration changes.