An Investigation of Finite-Sample Behavior of Confidence Interval Estimators

Abstract

We investigate the small-sample behavior and convergence properties of confidence interval estimators (CIEs) for the mean of a stationary discrete process. We consider CIEs arising from nonoverlapping batch means, overlapping batch means, and standardized time series, all of which are commonly used in discrete-event simulation. The performance measures of interest are the coverage probability, and the expected value and variance of the half-length. We use empirical and analytical methods to make detailed comparisons regarding the behavior of the CIEs for a variety of stochastic processes. All the CIEs under study are asymptotically valid; however, they are usually invalid for small sample sizes. We find that for small samples, the bias of the variance parameter estimator figures significantly in CIE coverage performance—the less bias the better. A secondary role is played by the marginal distribution of the stationary process. We also point out that some CIEs require fewer observations before manifesting the properties for CIE validity.

Convergence Properties of the Batch Means Method for Simulation Output Analysis

“…Kang and Goldsman (1990) establish (19) when the underlying process X i is a stationary autoregressive-moving process of finite order (that is, ARMA(p,q) with p q < ) whose white noise process has a symmetric marginal with finite mean, variance, and skewness. Moreover, Sargent, Kang, and Goldsman (1992) establish (19) when X i is stationary multivariate normal and we use k = 2 batches.…”

“…Song and Schmeiser (1995) studied three of these processes in their investigation of the optimal batch size for which S 2 m k /k is a minimum-mean-squared-error estimator of Var Y m k . For some of our test processes, Sargent, Kang, and Goldsman (1992) provided analytic and Monte Carlo results characterizing the expected halflength and coverage probability of confidence intervals delivered by the NOBM method and by the methods of overlapping batch means and standardized time series; moreover they plotted empirical distributions of the NOBM t-statistic for k = 8 batches and for various batch sizes. In Subsections 4.1-4.4 below, we give analytic expressions for the covariance function and SSVC of each of the selected test processes.…”

On the Distribution of Throughput of Transfer Lines

“…Law [135] shows that the sample mean of the first delays for the M and Sargent,Kang, and Goldsman [206].…”

Selecting Stopping Rules for Confidence Interval Procedures

“…Law and Kelton [1982] conducted a survey of sequential procedures, and found some rules that performed favorably, although they acknowledged that small sample sizes could lead to a loss in the coverage. Fixed-sample analysis of confidence interval coverage for small samples was investigated in Sargent et al [1992], where the authors study the importance of having an unbiased variance estimator. Finite-sample analytical results for i.i.d.…”