Statistical Inference using Bootstrap Confidence Intervals

Abstract

Bootstrap confidence intervals provide a way of quantifying the uncertainties in the inferences that can be drawn from a sample of data. The idea is to use a simulation, based on the actual data, to estimate the likely extent of sampling error. Michael Wood explains how simple bootstrapping works and explores some of its advantages.

Quantifying error in OSCE standard setting for varying cohort sizes: A resampling approach to measuring assessment quality

“…Whilst the technical challenge of carrying out the repeated calculations through a programing approach in R is initially considerable, the resampling approach is conceptually simple once one is prepared to accept that a single set of assessment data actually contains an very large number of 'samples' (formally, resamples) for a fixed cohort size (Wood, 2004). The method also rests on less restrictive assumptions than do alternative approaches based on arguably more challenging RMSE and/or generalizability formulae.…”

“…For example, in a small cohort ten station OSCE (n=59), Wood et al (2006) calculate the standard error of the regression line within each station using an established regression-based formula (Draper & Smith, 1998, p. 80) and compare this to that of the modified borderline group (MBG) method (Cizek & Bunch, 2007, pp. 112-116;Wood et al, 2006 In order to estimate the standard error of station-level and overall pass marks as set by the BRM, this paper analyses four sets of recent OSCE data from two medical schools and uses an innovative application of bootstrapping/resampling methods (Boos & Stefanski, 2010;Efron & Tibshirani, 1994;Wood, 2004). A key question in this research is how these errors in pass marks vary with cohort size -as cohort sizes get smaller, at what point do these errors become indefensibly large?…”

“…pass marks, reliability figures) and hence can estimate the variation in each of these by looking at the resulting distribution of their values over a large number (often 1000) of resamples. The power of this approach is that repeated resampling from the sample mimics (hypothetical) resampling from the population, and hence gives otherwise unobtainable insight into the sampling variation of the statistic of interest (Boos & Stefanski, 2010;Wood, 2004). In addition, the properties required of the data under bootstrapping (e.g.…”

“…In addition, the properties required of the data under bootstrapping (e.g. distributional assumptions) are usually less restrictive compared to more conventional methods (Wood, 2004). Table 1 shows the four sets of OSCE data that are used in the study.…”

Fair Statistical Communication in HCI

“…Bootstrapping is a very useful method that has not received enough attention (Kirby and Gerlanc, 2013;Wood, 2004Wood, , 2005. Briefly, it consists of generating many alternative datasets from the experimental data by randomly drawing observations with replacement.…”