Graphical methods for the design and analysis of simulation experiments

In this paper we develop some intuitive graphical methods for designing, running, and analyzing simulation experiments. We first concentrate on the analysis of output plots from a single run; the concept of a standardized simulation output plot is presented and its use illustrated. The remainder of the paper involves the design of experiments that may involve several (sequential) simulation runs. Experimental design discussions in simulation often focus on special topics unique to the field. Control over random variables permits design techniques such as antithetic variates and common random number streams. Yet many of the general topics of experiment design including confounding, fractional designs, and sample size determination are important in a simulation setting. The second part of this tutorial will focus on these general topics and present some graphical tools for generating experiment designs: causal diagrams and multidimensional point plots.