Optimal Stopping and Experimental Design

The problem of finding Bayes solutions for sequential experimental design problems motivates the study of the following type of one-person sequential game. If the game is stopped at any stage a, the loss to the player is the value of a random variable (rv), say Za. If the player chooses to continue the game, he can select the next rv to be observed from a class of rv's available at that stage, thus bringing the game to one of the stages succeeding stage a. (The class of all stages can be pictured as a "tree".) At this stage the player can again choose to stop, accepting the value of the chosen rv as his loss, or he may continue by selecting one of the class of rv's now available for the next observation. The player is required to stop sometime, and his decisions at any stage must depend only on information available at that stage. A model for this situation is given in Section 4. Control variables, which correspond to stopping variables in the usual formulation of sequential games, are defined which can be used by the player to decide whether to stop or not at any stage and, if he continues, which rv to observe next. A general characterization of control variables that minimize expected loss is given, and existence of such optimal control variables is proved under conditions applicable to statistical problems. The application to finding Bayes solutions to sequential experimental design problems is given in Section 5. As a preliminary to the discussion on control variables, Section 3 provides a study of the theory of optimal stopping variables. Let {Zn, Fn, n ≥ 1} be a stochastic process on a probability space (Ω, F, P) with points ω. A stopping variable (sv) is a rv t with values in {1, 2, ⋯, ∞} such that $t < \infty$ a.e. and {t = n} ε Fnfor each n. For any such sv t, a rv Ztis defined by \begin{align*}Z_t(\omega) &= Z_n(\omega),\quad\text{if} t(\omega) = n, \\ &= \infty,\quad\text{if} t(\omega) = \infty.\end{align*} It is convenient to think of Znas the loss after n plays in a one-person sequential game and to consider the σ-field Fnas representing the knowledge of the past after n plays. The problem of finding a strategy for stopping the game to minimize the expected loss corresponds to finding a minimizing sv, i.e., one which minimizes EZtamong the class of all sv's t. The main results in Section 3 are new characterizations of Snell's solution in [12] to the problem of optimal stopping which generalized the well-known Arrow-Blackwell-Girshick theory in