A Common Control Group - Optimising the Experiment Design to Maximise Sensitivity

Bate, Simon T.; Karp, Natasha A.

doi:10.1371/journal.pone.0114872

Cited by 18 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Pre‐registration of experimental design and intended methods of analysis is not yet common in our sector. We agree that optimally unbalanced groups can lead to improved sensitivity and power when the a priori decision is made to analyse them without ANOVA and with (for instance) Dunnett's tests back to a single comparator rather than all pairwise comparisons (Bate and Karp, ). However, in our experience, reviewers and editors often cannot tell whether experiments with unbalanced groups result from planned excellent design or unconsidered design and inadequate transparency, with attrition unreported and exclusions undeclared. Some investigators do not undertake blinded and randomized studies, and animals are added to, or removed from, the study after preliminary analysis.…”

mentioning

confidence: 69%

Clarification of the basis for the selection of requirements for publication in the British Journal of Pharmacology

Curtis

Ashton

Moon

et al. 2018

British J Pharmacology

View full text Add to dashboard Cite

In 2015 and 2018, the British Journal of Pharmacology (BJP) published guidelines on experimental design and analysis (Curtis et al., 2015. The intention was to improve the credibility of papers published in BJP by the simplest means possible. It is all very well for a journal to elaborate a framework of best practice, with lengthy explanations for each issue considered, but if authors, reviewers and editors fail to adopt the framework because it is too complex or nuanced, then we fail as a journal. Consequently, unlike most other journals (Williams et al., 2018), BJP has opted for firm rules about a small number of issues, rather than generalized and lengthy 'best practice advice'. We focused on inconsistent reporting of P values (e.g. P < 0.05, P = exact value, P < different values), persistent and unjustified use of n = 3 (or fewer), grossly unequal group sizes and an absence of randomization and blinding (each of which typically occurs together in many papers) that are particular problems in our sector and contribute to the failed replication that is undermining the credibility of preclinical research. We received two letters that criticize some of our guidance and have written an itemized reply below.First, we make a general point. Most of the BJP guidelines are 'conventions', that is, pragmatic solutions to practical challenges. This is particularly relevant to BJP's requirements for group size selection. Setting n = 5 as the minimum allowable for comparing groups by statistical analysis (the 'n = 5 rule') is clearly a convention. We are not claiming n = 5 is sufficient and necessary for all studies. In some studies, group sizes much larger than n = 5 are necessary to reduce the risk of false findings, whereas in other studies, where the control outcome has been established repeatedly in previous published work, group sizes of fewer than n = 5 may be sufficient. In the main, BJP publishes papers on new drugs, or using new transgenic animals, or evaluating variables that have not been evaluated previously, often a combination of all three. Novelty is the key. When work is novel, it is extraordinarily rare for an author to include in their Methods section a clear statement that the data are known to be drawn from a normally distributed population (the necessary prerequisite for the type of parametric analysis typically undertaken) or that they have undertaken sample size calculations a priori that indicate that n = X would be adequate for their design. Consequently, it seems that deciding on an appropriate group size is done by after-the-fact power analysis using the data generated by a study to justify the group size used in the study (as opposed to a priori power analysis) or by 'informed judgement' (guesswork). Moreover, 'group sizes as small as possible' is normally the guiding principle. The resultant problem is that studies are often favourably treated by peer review if sufficiently novel, with no questioning of group size selection. This is not a problem that can be ignored. Most statistical software pr...

show abstract

mentioning

confidence: 69%

Clarification of the basis for the selection of requirements for publication in the British Journal of Pharmacology

Curtis

Ashton

Moon

et al. 2018

British J Pharmacology

View full text Add to dashboard Cite

show abstract

“…Some studies used larger animals, however, these studies used a smaller number of animals per group, possibly as a result of the cost, due to its not being practical for the majority of research installations, as well as presenting difficulties with handling during the realization of procedures [65]. In this context, the reduction in the number of experimental animals may be a complicating factor in research due to increased risk of obtaining inconclusive results [66].…”

Section: Discussionmentioning

confidence: 99%

Does antibiotic use accelerate or retard cutaneous repair? A systematic review in animal models

et al. 2019

View full text Add to dashboard Cite

BackgroundThe presence of infections is one of the main factors that leads to delays in healing or non-closure of cutaneous wounds. Although the goal of antibiotic use is to treat or prevent infection, there is currently no agreement on the effectiveness of these products.AimThe aim of this study was to evaluate the efficacy of antibiotic use during the healing process of skin wounds in animal models not intentionally infected, as well as to analyze the advances and limitations of the studies carried out in this field.Main methodsThis systematic review was performed according to the PRISMA guidelines, using a structured search on the MedLine (PubMed) and Scopus platforms to retrieve studies published until August 29, 2018, 13:35p.m. The studies included were limited to those that used excision or incision wound models and that were not intentionally infected. The data for the animal models, antibiotic used, and the main results of the studies were extracted, and compared where possible. Bias analysis and methodological quality assessments were examined through the SYRCLE’s Risk of Bias tool.Key findingsTwenty-seven studies were selected. Overall, the effects of the antibiotic on the wound decreased inflammatory cell infiltration and promoted an increased number of fibroblasts, extracellular matrix constituents, re-epithelialization and tissue strength. A great deal of important information about the methodology was not presented, such as: the statistical analysis used, the animal model (sex and age), antibiotic dosage, blinding and randomization of the animals chosen.SignificanceBased on the results found, we believe that antibiotic therapy can be considered a viable alternative for the treatment of cutaneous wounds. However, current evidence obtained from the methodological quality analysis points towards a high risk of bias. This is due to the incomplete characterization of the experimental design and treatment protocol, which compromises the reproducibility of the studies.

show abstract

“…Haimez 2002;Aban and George 2015;Singh et al 2016). Bate and Karp (2014) have looked more closely at the question of relative group sizes, and they show that the traditional, balanced approach is indeed the best experimental design when all pairwise comparisons are planned. However, when the planned comparisons are of several treatment groups with a single control group, there is a small gain in sensitivity by having relatively more animals in the control group.…”

Section: What Group Size For Control Groups?mentioning

confidence: 99%

Out of Control? Managing Baseline Variability in Experimental Studies with Control Groups

Moser¹

2019

Good Research Practice in Non-Clinical Pharmacology and Biomedicine

View full text Add to dashboard Cite

Control groups are expected to show what happens in the absence of the intervention of interest (negative control) or the effect of an intervention expected to have an effect (positive control). Although they usually give results we can anticipate, they are an essential component of all experiments, both in vitro and in vivo, and fulfil a number of important roles in any experimental design. Perhaps most importantly they help you understand the influence of variables that you cannot fully eliminate from your experiment and thus include them in

show abstract

A Common Control Group - Optimising the Experiment Design to Maximise Sensitivity

Cited by 18 publications

References 18 publications

Clarification of the basis for the selection of requirements for publication in the British Journal of Pharmacology

Clarification of the basis for the selection of requirements for publication in the British Journal of Pharmacology

Does antibiotic use accelerate or retard cutaneous repair? A systematic review in animal models

Out of Control? Managing Baseline Variability in Experimental Studies with Control Groups

Contact Info

Product

Resources

About