Detecting Initialization Bias in Simulation Output

Abstract

A general approach to testing for initialization bias in the mean of a simulation output series is presented. The output is transformed into a standardized test sequence that can be contrasted with a known limiting stochastic process. This transformation requires very little computation and the asymptotic theory is applicable to a wide variety of simulations. An initialization bias test is developed and several examples of its application are presented.

Analysis of the behavior of the transient period in non-terminating simulations

“…A second way, called stochastic (random) initialization, tries to estimate the steady-state probability distribution of the process, possibly from pilot runs, and then uses this estimated distribution to sample the initial conditions. Madansky (1976) shows that initializing an M/M/1 queue in empty and idle state, which is Gafarian et al (1978), Wilson and Pritsker (1978a,b), Chance (1993), Fishman (1972, Kleijnen (1984), Law (1984), Nelson (1990Nelson ( , 1992, Cash et al (1992), Ma and Kochhar (1993) Intelligent initialization Deterministic initialization Madansky (1976), Kelton and Law (1985), Kelton (1985), Murray and Kelton (1988a) Stochastic initialization Kelton (1989), Murray (1988), Murray and Kelton (1988b) Schruben (1981Schruben ( , 1982, Schruben et al (1983), Goldsman et al (1994), Vassilacopoulus (1989) Analytical techniques Kelton and Law (1983), Asmussen et al (1992), Gallagher et al (1996), White (1997), Spratt (1998), White et al (2000) the mode of the number-in-system distribution, minimizes the MSE of the point estimate. For M/M/s, M/E m /1, M/E m /2, and E m /M/2 queues, Kelton and Law (1985), Kelton (1985), and Murray and Kelton (1988a) find that initializing in a state at least as congested as the steady-state mean (as opposed to the mode) induces shorter transient periods.…”

“…However, if the bias decays slowly, it becomes harder for the tests to detect the bias. Ma and Kochhar (1993) compare the test procedures of Schruben (1982) and Vassilacopoulus (1989), using sequences with known transient distributions. Their results indicate that both tests are powerful, but they recommend VassilacopoulusÕs test due to its ease of implementation.…”

“…When the resulting statistics are plotted, the truncation point is chosen to be the point where the graph flattens out. Schruben (1982) develops a very general procedure for univariate output based on standardized time-series. This procedure is the basic building block of techniques discussed by Schruben et al (1983) and Goldsman et al (1994), which we call repetitive hypothesis testing.…”

“…This is called the initial transient, initialization bias, or the start-up problem in the simulation literature. Several techniques have been proposed to remedy this problem (see, for example, Kelton, 1989;Kelton and Law, 1983;Schruben, 1982;Schruben et al, 1983;Goldsman et al, 1994;Vassilacopoulus, 1989;Welch, 1982;White, 1997).…”

Automating warm-up length estimation

“…Kimbler's Double Exponential Smoothing Method Kimbler and Knight (1987) Euclidean Distance (ED) Method Lee et al (1997) Neural Networks (NN) Method Lee et al (1997) Statistical Goodness-Of-Fit Test Pawlikowski (1990) Algorithm for a Static Dataset (ASD) Bause and Eickhoff (2003) Algorithm for a Dynamic Dataset (ADD) Bause and Eickhoff (2003) Kelton and Law Regression Method Kelton and Law (1983), Law (1983), Kimbler and Knight (1987), Pawlikowski (1990), Roth and Josephy (1993), Roth (1994), Gallagher et al (1996), Law and Kelton (2000), Linton and Harmonosky (2002) Glynn & Iglehart Bias Deletion Rule Glynn and Iglehart (1987) Wavelet-Based Spectral Method (WASSP) Lada et al (2003), Lada et al (2004), Lada and Wilson (2006) Queueing Approximations Method (MSEASVT) Rossetti and Delaney (1995) Chaos Theory Methods (methods M1 and M2) Lee and Oh (1994) Kalman Filter Method Gallagher et al (1996), Law and Kelton (2000) Randomisation Tests for Initialisation Bias Yucesan (1993), Mahajan and Ingalls (2004) Initialisation bias tests Schruben's Maximum Test (STS) Schruben (1982), Law (1983), Schruben et al (1983), Yucesan (1993), Ockerman and Goldsman (1999), Law andKelton (2000) Schruben's Modified Test Schruben (1982), , Law (1983), White et al(2000), Law and Kelton (2000) Optimal Test (Brownian bridge process) …”