BackgroundIn 2004, a review of pilot studies published in seven major medical journals during 2000-01 recommended that the statistical analysis of such studies should be either mainly descriptive or focus on sample size estimation, while results from hypothesis testing must be interpreted with caution. We revisited these journals to see whether the subsequent recommendations have changed the practice of reporting pilot studies. We also conducted a survey to identify the methodological components in registered research studies which are described as 'pilot' or 'feasibility' studies. We extended this survey to grant-awarding bodies and editors of medical journals to discover their policies regarding the function and reporting of pilot studies.MethodsPapers from 2007-08 in seven medical journals were screened to retrieve published pilot studies. Reports of registered and completed studies on the UK Clinical Research Network (UKCRN) Portfolio database were retrieved and scrutinized. Guidance on the conduct and reporting of pilot studies was retrieved from the websites of three grant giving bodies and seven journal editors were canvassed.Results54 pilot or feasibility studies published in 2007-8 were found, of which 26 (48%) were pilot studies of interventions and the remainder feasibility studies. The majority incorporated hypothesis-testing (81%), a control arm (69%) and a randomization procedure (62%). Most (81%) pointed towards the need for further research. Only 8 out of 90 pilot studies identified by the earlier review led to subsequent main studies. Twelve studies which were interventional pilot/feasibility studies and which included testing of some component of the research process were identified through the UKCRN Portfolio database. There was no clear distinction in use of the terms 'pilot' and 'feasibility'. Five journal editors replied to our entreaty. In general they were loathe to publish studies described as 'pilot'.ConclusionPilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. Authors should be aware of the different requirements of pilot studies, feasibility studies and main studies and report them appropriately. Authors should be explicit as to the purpose of a pilot study. The definitions of feasibility and pilot studies vary and we make proposals here to clarify terminology.

Sample size justification is an important consideration when planning a clinical trial, not only for the main trial but also for any preliminary pilot trial. When the outcome is a continuous variable, the sample size calculation requires an accurate estimate of the standard deviation of the outcome measure. A pilot trial can be used to get an estimate of the standard deviation, which could then be used to anticipate what may be observed in the main trial. However, an important consideration is that pilot trials often estimate the standard deviation parameter imprecisely. This paper looks at how we can choose an external pilot trial sample size in order to minimise the sample size of the overall clinical trial programme, that is, the pilot and the main trial together. We produce a method of calculating the optimal solution to the required pilot trial sample size when the standardised effect size for the main trial is known. However, as it may not be possible to know the standardised effect size to be used prior to the pilot trial, approximate rules are also presented. For a main trial designed with 90% power and two-sided 5% significance, we recommend pilot trial sample sizes per treatment arm of 75, 25, 15 and 10 for standardised effect sizes that are extra small (≤0.1), small (0.2), medium (0.5) or large (0.8), respectively.

No abstract

on behalf of the REEACT Team ABSTRACT Study queStionHow effective is supported computerised cognitive behaviour therapy (cCBT) as an adjunct to usual primary care for adults with depression? MethodSThis was a pragmatic, multicentre, three arm, parallel randomised controlled trial with simple randomisation. Treatment allocation was not blinded. Participants were adults with symptoms of depression (score ≥10 on nine item patient health questionnaire, PHQ-9) who were randomised to receive a commercially produced cCBT programme ("Beating the Blues") or a free to use cCBT programme (MoodGYM) in addition to usual GP care. Participants were supported and encouraged to complete the programme via weekly telephone calls. Control participants were offered usual GP care, with no constraints on the range of treatments that could be accessed. The primary outcome was severity of depression assessed with the PHQ-9 at four months. Secondary outcomes included health related quality of life (measured by SF-36) and psychological wellbeing (measured by CORE-OM) at four, 12, and 24 months and depression at 12 and 24 months. Study anSwer and liMitationSParticipants offered commercial or free to use cCBT experienced no additional improvement in depression compared with usual GP care at four months (odds ratio 1.19 (95% confidence interval 0.75 to 1.88) for Beating the Blues v usual GP care; 0.98 (0.62 to 1.56) for MoodGYM v usual GP care). There was no evidence of an overall difference between either programme compared with usual GP care (0.99 (0.57 to 1.70) and 0.68 (0.42 to 1.10), respectively) at any time point. Commercially provided cCBT conferred no additional benefit over free to use cCBT or usual GP care at any follow-up point. Uptake and use of cCBT was low, despite regular telephone support. Nearly a quarter of participants (24%) had dropped out by four months. The study did not have enough power to detect small differences so these cannot be ruled out. Findings cannot be generalised to cCBT offered with a much higher level of guidance and support.

ObjectivesDesigning studies with an internal pilot phase may optimise the use of pilot work to inform more efficient randomised controlled trials (RCTs). Careful selection of preagreed decision or ‘progression’ criteria at the juncture between the internal pilot and main trial phases provides a valuable opportunity to evaluate the likely success of the main trial and optimise its design or, if necessary, to make the decision not to proceed with the main trial. Guidance on the appropriate selection and application of progression criteria is, however, lacking. This paper outlines the key issues to consider in the optimal development and review of operational progression criteria for RCTs with an internal pilot phase.DesignA structured literature review and exploration of stakeholders' opinions at a Medical Research Council (MRC) Hubs for Trials Methodology Research workshop. Key stakeholders included triallists, methodologists, statisticians and funders.ResultsThere is considerable variation in the use of progression criteria for RCTs with an internal pilot phase, although 3 common issues predominate: trial recruitment, protocol adherence and outcome data. Detailed and systematic reporting around the decision-making process for stopping, amending or proceeding to a main trial is uncommon, which may hamper understanding in the research community about the appropriate and optimal use of RCTs with an internal pilot phase. 10 top tips for the development, use and reporting of progression criteria for internal pilot studies are presented.ConclusionsSystematic and transparent reporting of the design, results and evaluation of internal pilot trials in the literature should be encouraged in order to facilitate understanding in the research community and to inform future trials.

Objective To investigate health researchersÕ attitudes to involving the public in research.Background Public involvement in research is encouraged by the Department of Health in the UK. Despite this, the number of health researchers actively involving the public in research appears to be limited. There is little research specifically addressing the attitudes of health researchers towards involving the public: how they interpret the policy, what motivates and de-motivates them and what their experiences have been to date.

National Institute for Health Research Health Technology Assessment programme.

Objective: To provide the first valid and reliable estimate of the health status of Gypsies and Travellers in England by using standardised instruments to compare their health with that of a UK resident non-Traveller sample, drawn from different socioeconomic and ethnic groups, matched for age and sex. Design: Epidemiological survey, by structured interview, of quota sample and concurrent age-sex-matched comparators. Setting: The homes or alternative community settings of the participants at five study locations in England. Participants: Gypsies and Travellers of UK or Irish origin (n = 293) and an age-sex-matched comparison sample (n = 260); non-Gypsies or Travellers from rural communities, deprived inner-city White residents and ethnic minority populations. Results: Gypsies and Travellers reported poorer health status for the last year, were significantly more likely to have a long-term illness, health problem or disability, which limits daily activities or work, had more problems with mobility, self-care, usual activities, pain or discomfort and anxiety or depression as assessed using the EuroQol-5D health utility measure, and a higher overall prevalence of reported chest pain, respiratory problems, arthritis, miscarriage and premature death of offspring. No inequality was reported in diabetes, stroke and cancer. Conclusions: Significant health inequalities exist between the Gypsy and Traveller population in England and their non-Gypsy counterparts, even when compared with other socially deprived or excluded groups, and with other ethnic minorities.