What is a pilot or feasibility study? A review of current practice and editorial policy

Abstract

BackgroundIn 2004, a review of pilot studies published in seven major medical journals during 2000-01 recommended that the statistical analysis of such studies should be either mainly descriptive or focus on sample size estimation, while results from hypothesis testing must be interpreted with caution. We revisited these journals to see whether the subsequent recommendations have changed the practice of reporting pilot studies. We also conducted a survey to identify the methodological components in registered research studies which are described as 'pilot' or 'feasibility' studies. We extended this survey to grant-awarding bodies and editors of medical journals to discover their policies regarding the function and reporting of pilot studies.MethodsPapers from 2007-08 in seven medical journals were screened to retrieve published pilot studies. Reports of registered and completed studies on the UK Clinical Research Network (UKCRN) Portfolio database were retrieved and scrutinized. Guidance on the conduct and reporting of pilot studies was retrieved from the websites of three grant giving bodies and seven journal editors were canvassed.Results54 pilot or feasibility studies published in 2007-8 were found, of which 26 (48%) were pilot studies of interventions and the remainder feasibility studies. The majority incorporated hypothesis-testing (81%), a control arm (69%) and a randomization procedure (62%). Most (81%) pointed towards the need for further research. Only 8 out of 90 pilot studies identified by the earlier review led to subsequent main studies. Twelve studies which were interventional pilot/feasibility studies and which included testing of some component of the research process were identified through the UKCRN Portfolio database. There was no clear distinction in use of the terms 'pilot' and 'feasibility'. Five journal editors replied to our entreaty. In general they were loathe to publish studies described as 'pilot'.ConclusionPilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. Authors should be aware of the different requirements of pilot studies, feasibility studies and main studies and report them appropriately. Authors should be explicit as to the purpose of a pilot study. The definitions of feasibility and pilot studies vary and we make proposals here to clarify terminology.

Alcohol and disadvantaged men: A feasibility trial of an intervention delivered by mobile phone

“…The main purpose of the follow up was to determine whether the outcomes could be assessed, and complete data was obtained on those followed up. Following guidance for feasibility studies 31, 32, 62, 63, 64, no formal statistical testing for effectiveness was carried out. Inspection of the data (Table 2) shows that men in the intervention group reduced the frequency of their binge drinking (8 or more units in a session) by slightly more than men in the control group.…”

“…Feasibility/pilot studies are essential when planning a full trial, providing the opportunity to test and improve all study procedures 31, 32. The aim is to ensure that design weaknesses, technical problems and methodological flaws do not compromise the full trial 33.…”

Virtual small groups for weight management: an innovative delivery mechanism for evidence-based lifestyle interventions among obese men

“…No studies to date have examined provider-to-multiple end-user interactions where the virtual meeting "room" is accessed from several different locations (as opposed to just two). In this pilot study [50], entitled "Virtual Small Groups for an Innovative and Technological Approach to Healthy Lifestyle" (VITAL)", we tested group VC for weight loss. The primary aim of this randomized controlled pilot study was to use group VC to deliver an evidence-based lifestyle intervention to virtual small groups and to compare the change in body weight and BMI from baseline to 3 months between intervention and control groups.…”

UPBEAT-UK: a programme of research into the relationship between coronary heart disease and depression in primary care patients

“…188 Often there is no clear distinction in the literature between pilot and feasibility studies. 189 A literature review found that definitions of feasibility and pilot studies vary, but studies labelled 'feasibility' tend to use a flexible method compared with 'pilot' studies in practice.…”

“…For pilot studies such as ours it has been suggested that the analysis should mainly be descriptive or focus on the CI estimates. 187 Pilot studies are described by Arain et al 188 as a smaller version of a main study, to ensure components of the main study work well. They are 'focused on the processes of the main study' such as 'recruitment, randomisation, treatment, and follow-up'.…”