The importance of small samples in medical research

Abstract

Almost all bio-statisticians and medical researchers believe that a large sample is always helpful in providing more reliable results. Whereas this is true for some specific cases, a large sample may not be helpful in more situations than we contemplate because of the higher possibility of errors and reduced validity. Many medical breakthroughs have occurred with self-experimentation and single experiments. Studies, particularly analytical studies, may provide more truthful results with a small sample because intensive efforts can be made to control all the confounders, wherever they operate, and sophisticated equipment can be used to obtain more accurate data. A large sample may be required only for the studies with highly variable outcomes, where an estimate of the effect size with high precision is required, or when the effect size to be detected is small. This communication underscores the importance of small samples in reaching a valid conclusion in certain situations and describes the situations where a large sample is not only unnecessary but may even compromise the validity by not being able to exercise full care in the assessments. What sample size is small depends on the context.

Small Study Effects in Diagnostic Imaging Accuracy

“… 2 , 61 , 62 In contrast, smaller studies require less time and resources to conduct, which may be important when investigating novel technologies with an exploratory purpose but can be more prone to bias. 63 …”

“…2,61,62 In contrast, smaller studies require less time and resources to conduct, which may be important when investigating novel technologies with an exploratory purpose but can be more prone to bias. 63 Apart from publication bias, other potential explanations for small study effects have been identified. 9 These include other forms of reporting bias, such as selective outcome reporting and selective analysis reporting, true heterogeneity, or the presence of low-quality small studies producing greatly inflated effect sizes.…”

“…It is generally accepted that larger studies are more rigorous and well controlled and less prone to publication bias for several reasons, including the likelihood of being preregistered, the involvement of more experienced research teams, and the time and money invested . In contrast, smaller studies require less time and resources to conduct, which may be important when investigating novel technologies with an exploratory purpose but can be more prone to bias …”

The development of QERM scoring system for comprehensive assessment of the Quality of Empirical Research in Medicine - Part 1

“…The adequacy of different components of research can be enumerated as follows: (i) original, novel, and justified research question, (ii) sufficient resources for completing the research, (iii) measurable objectives, (iv) adequate intervention to achieve the stated objectives, (v) appropriate target population, (vi) adequate tools for collection of the data, (vii) no relevant variable missed, (viii) study design that takes care of confounders and interactions, (ix) sufficient sample size not discounting the advantages of small samples,[ 45 ] (x) representative sample, (xi) ethical and complete data collection, (xiii) appropriate analysis for the type of data and focused on the research question, (xiii) the results with sufficient reliability, and (xiv) valid reasons available if the research question is not fully answered. For details, see the Supplementary Material - 2.…”

A Bayesian Sample Size Estimation Procedure Based on a B-Splines Semiparametric Elicitation Method

“…However, in some cases, excessively large sample sizes are not necessary because a smaller number of patients may be useful in estimating the expected effect with the same precision. The tradeoff between a more and less conservative design depends on available prior information, agreement among experts, safety issues, etc [ 12 ].…”