A sequential experimental design procedure for the estimation of first- and second-order simulation metamodels

Simulation metamodels find application in the study of complex systems that cannot be solved analytically. These metamodels represent efficient tools for studying the characteristics of the more complicated simulation model, provide needed insight into the problems of computer model validation and verification, and allow for the prediction of both system performance and optimum operating conditions. This article presents a procedure for the construction of sequential simulation designs for the estimation of response surface metamodels. The first set of experiments is defined as a fractional two-level factorial design augmented with replicated center points. Information from these experiments is used to estimate the levels of the factorial design points that constitute the second stage of experimentation. If observations on this two-stage, first-order design suggest the presence of unfitted quadratic terms, a third set of observations corresponding to the axial portion of a central composite design is taken to allow for the estimation of a second-order metamodel. Two types of performance criteria are considered in the specification of the factor settings in the second and third stages: (1) minimizing errors associated with predicting the response variable and (2) minimizing errors involved with estimating the response function slopes. Additionally, three methods of assigning random number streams to the stochastic components of the simulation model are considered: (1) independent streams, (2) common streams, and (3) the assignment rule blocking strategy. An example illustrating the use of the sequential design procedure is presented, and a Monte Carlo study investigates the performance of the two variance reduction techniques (common streams and the assignment rule) relative to independent stream sets. Empirical results indicate a preference for the assignment rule strategy for the estimation of both first- and second-order metamodels.