Design and analysis of pilot studies: recommendations for good practice

Abstract

Pilot studies play an important role in health research, but they can be misused, mistreated and misrepresented. In this paper we focus on pilot studies that are used specifically to plan a randomized controlled trial (RCT). Citing examples from the literature, we provide a methodological framework in which to work, and discuss reasons why a pilot study might be undertaken. A well-conducted pilot study, giving a clear list of aims and objectives within a formal framework will encourage methodological rigour, ensure that the work is scientifically valid and publishable, and will lead to higher quality RCTs. It will also safeguard against pilot studies being conducted simply because of small numbers of available patients.

Assessing the efficacy and social acceptability of using hygienist-therapists as front-line clinicians

“…A sample size of 60 provided sufficient power to estimate a recruitment rate of 50% to within a one-sided 95% confidence interval of 10.62%. 33 …”

UPBEAT-UK: a programme of research into the relationship between coronary heart disease and depression in primary care patients

“…188 Often there is no clear distinction in the literature between pilot and feasibility studies. 189 A literature review found that definitions of feasibility and pilot studies vary, but studies labelled 'feasibility' tend to use a flexible method compared with 'pilot' studies in practice. Arain et al 188 suggest that feasibility studies are undertaken before a main study to 'estimate important parameters' for the design of a main study, such as the SD of the outcome measure for sample size calculation, characteristics and feasibility of the outcome measure, people's readiness to be recruited and randomised, number of eligible participants, adherence and follow-up rates.…”

“…Feasibility studies should 'not evaluate the outcome of interest' but instead be powered to estimate parameters such as recruitment or dropout rates. Feasibility or external pilot studies assess one or more of the following: 186,189 1. the feasibility of key processes, such as eligibility criteria, and their ascertainment, randomisation procedures and their acceptability, retention and refusal rates, failure/success rates, adherence, appropriateness of the primary outcome measure or the acceptability of questionnaires 2. potential time and resource problems, such as administrative issues, staff training, the willingness and capacity of involved clinicians or centres, process times, for example for postage, preparing equipment, dealing with contingencies such as material breakdown, clinician absence 3. potential human and data management problems, including space for personnel and data collection forms, data entry and basic properties of the data such as missingness or variability 4. acceptability and safety of the intervention, dose, response, estimated treatment effect and estimated variance of the treatment effect for sample size calculation 5. the integrity of the study protocol.…”

Alternative Approach for Conducting Construction Management Research: Quasi-Experimentation

“…Pre-tests are carried out on a few preliminary subjects to see how the experimental procedures affect them. Analysis of the pretesting reveals the problems of the design, equipment failure, ambiguous instruction, and other aspects in the stage in the research where corrections and adjustments can still be made (Lancaster et al 2004). Before the start of the 'production', subjects must be recruited and the investigators must obtain the subjects' informed consent to participate.…”