The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials

Jakobsen, Janus Christian; Gluud, Christian; Winkel, Per; Lange, Theis; Wetterslev, Jørn

doi:10.1186/1471-2288-14-34

Cited by 161 publications

(195 citation statements)

References 81 publications

Supporting

Mentioning

190

Contrasting

Order By: Relevance

“…4, p. 424-438 low standardised methodologies 5,11,27 . Meta-analyses conducted without a protocol run the risk of systematic, design, and random errors, which may cloud our judgement on benefits and harms of interventions, and makes it difficult to design future trials validly 26,[28][29][30] . Systematic reviews with meta-analysis of several small RCTs compared to a single, large RCT…”

Section: Search Strategy and Selection Criteriamentioning

confidence: 99%

“…Evidence from a well-conducted systematic review of several RCTs with low risk of bias therefore generally represents a higher level of evidence compared to the results from a single RCT [11][12][13][14]29,30 . It also appears intuitively evident that inclusion of all available data from all RCTs with low risks of bias ever conducted, should be treated as a higher level of evidence compared to the data from one single RCT 13,30 .…”

Section: Search Strategy and Selection Criteriamentioning

confidence: 99%

“…In several cases, drugs have been implemented based only on surrogate results even when similar drugs existed with proof of benefits on patient-centred outcomes 16 . Several drugs are advertised based on surrogate outcomes even though they have no effect or detrimental effects on patient-centred outcomes 13,16,29,30 . RCTs ought to assess if an intervention is safe and effective.…”

Section: Abuse Of Surrogate Outcomesmentioning

confidence: 99%

“…To reduce the required sample size, RCTs often adopt composite outcomes 29,61,62 . However, composite outcomes make it difficult to interpret the clinical significance of the results 29,61 .…”

Section: Abuse Of Composite Outcomesmentioning

confidence: 99%

“…However, composite outcomes make it difficult to interpret the clinical significance of the results 29,61 . Any benefit on a composite outcome may be presumed to relate to all its components 61 , but evidence shows that intervention effects on composite outcomes often apply to a single component, most likely the less relevant 61,62 .…”

Section: Abuse Of Composite Outcomesmentioning

confidence: 99%

See 4 more Smart Citations

Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Garattini

Jakobsen

Wetterslev

et al. 2016

European Journal of Internal Medicine

Self Cite

103

View full text Add to dashboard Cite

Abstract.Using the best quality of clinical research evidence is essential for choosing the right treatment for patients. How to identify the best research evidence is, however, difficult. In this narrative review we summarise these threats and describe how to minimise them. Pertinent literature was considered through literature searches combined with personal files. Treatments should generally not be chosen based only on evidence from observational studies or single randomised clinical trials. Systematic reviews with meta-analysis of all identifiable randomised clinical trials with Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessment represent the highest level of evidence. Even though systematic reviews are trust worthier than other types of evidence, all levels of the evidence hierarchy are under threats from systematic errors (bias); design errors (abuse of surrogate outcomes, composite outcomes, etc.); and random errors (play of chance). Clinical research infrastructures may help in providing larger and better conducted trials. Trial Sequential Analysis may help in deciding when there is sufficient evidence in meta-analyses. If threats to the validity of clinical research are carefully considered and minimised, research results will be more valid and this will benefit patients and heath care systems. Sažetak. Primjena najkvalitetnijih dokaza kliničkih istraživanja ključna je u odabiru ispravnog liječenja pacijenata. No, način na koji će se odabrati najbolji dokazi predstavlja često poteškoću. Ovim preglednim člankom prikazujemo opasnosti navedenog odabira, kao i načine kako ih umanjiti. Relevantni izvori razmatrani su pretragom literature u kombinaciji s osobnim datotekama. Izbor liječenja uglavnom se ne bi smio temeljiti isključivo na opservacijskim ili pojedinačnim randomiziranim kliničkim studijama. Sustavni pregledi s metaanalizom svih identificiranih randomiziranih kliničkih studija procijenjenih sustavom stupnjevanja procjene, razvoja i evalua cije preporuka (engl. Grading of Recommendations Assessment, Development and Evaluation; GRADE) predstavljaju najvišu razinu dokaza. Iako su sustavni pregledi pouzdaniji od drugih vrsta dokaza, sve razine hijerarhije dokaza ugrožene su sustavnim pogreškama (engl. bias); pogreškama dizajna studije (zloupotreba surogatnih ishoda, složenih ishoda itd.) i slučajnim pogreškama (igra slučaja). Kliničke istraživačke infrastrukture mogu pomoći u pružanju većih i adekvatnije provedenih ispitivanja. Sekvencijska analiza studija može pomoći pri odlučivanju kada postoji dovoljna razina dokaza u metaanalizama. Ako se prijetnje valjanosti kliničkih istraživanja pažljivo razmatraju i minimiziraju, rezultati istraživanja bit će vrjedniji i korisniji pacientima i zdravstvenim sustavima.Ključne riječi: klinička praksa temeljena na dokazima; medicina temeljena na dokazima; meta-analiza; randomizirana klinička studija; sekvencijska analiza studija; sustavni pregled

show abstract

Section: Search Strategy and Selection Criteriamentioning

confidence: 99%

Section: Search Strategy and Selection Criteriamentioning

confidence: 99%

Section: Abuse Of Surrogate Outcomesmentioning

confidence: 99%

Section: Abuse Of Composite Outcomesmentioning

confidence: 99%

Section: Abuse Of Composite Outcomesmentioning

confidence: 99%

See 3 more Smart Citations

Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Garattini

Jakobsen

Wetterslev

et al. 2016

European Journal of Internal Medicine

Self Cite

103

View full text Add to dashboard Cite

show abstract

Efficacy and Safety of Flibanserin for the Treatment of Hypoactive Sexual Desire Disorder in Women

et al. 2016

View full text Add to dashboard Cite

, the US Food and Drug Administration (FDA) approved flibanserin as a treatment for hypoactive sexual desire disorder (HSDD) in premenopausal women, despite concern about suboptimal risk-benefit trade-offs. OBJECTIVE To conduct a systematic review and meta-analysis of randomized clinical trials assessing efficacy and safety of flibanserin for the treatment of HSDD in women. DATA SOURCES Medical databases (among others, Embase, Medline, Psycinfo) and trial registries were searched from inception to June 17, 2015. Reference lists of retrieved studies were searched for additional publications. STUDY SELECTION Randomized clinical trials assessing treatment effects of flibanserin in premenopausal and postmenopausal women were eligible. No age, language, or date restrictions were applied. Abstract and full-text selection was done by 2 independent reviewers. DATA EXTRACTION AND SYNTHESIS Data were extracted by one reviewer and checked by a second reviewer. Results were pooled using 2 approaches depending on the blinding risk of bias. MAIN OUTCOMES AND MEASURES Primary efficacy outcomes included number of satisfying sexual events (SSEs), eDiary sexual desire, and Female Sexual Function Index (FSFI) desire. Safety outcomes included, among others, 4 common adverse events (AEs): dizziness, somnolence, nausea, and fatigue. RESULTS Five published and 3 unpublished studies including 5914 women were included. Pooled mean differences for SSE change from baseline were 0.49 (95% CI, 0.32-0.67) between 100-mg flibanserin and placebo, 1.63 (95% CI, 0.45-2.82) for eDiary desire, and 0.27 (95% CI, 0.17-0.38) for FSFI desire. The risk ratio for study discontinuation due to AEs was 2.19 (95% CI, 1.50-3.20). The risk ratio for dizziness was 4.00 (95% CI, 2.56-6.27) in flibanserin vs placebo, 3.97 (95% CI, 3.01-5.24) for somnolence, 2.35 (95% CI, 1.85-2.98) for nausea, and 1.64 (95% CI, 1.27-2.13) for fatigue. Women's mean global impression of improvement scores indicated minimal improvement to no change. CONCLUSIONS AND RELEVANCE Treatment with flibanserin, on average, resulted in one-half additional SSE per month while statistically and clinically significantly increasing the risk of dizziness, somnolence, nausea, and fatigue. Overall, the quality of the evidence was graded as very low. Before flibanserin can be recommended in guidelines and clinical practice, future studies should include women from diverse populations, particularly women with comorbidities, medication use, and surgical menopause.

show abstract

Hypothermia vs Normothermia in Patients With Cardiac Arrest and Nonshockable Rhythm

Taccone,

Dankiewicz,

Cariou

et al. 2024

JAMA Neurol

Self Cite

View full text Add to dashboard Cite

ImportanceInternational guidelines recommend body temperature control below 37.8 °C in unconscious patients with out-of-hospital cardiac arrest (OHCA); however, a target temperature of 33 °C might lead to better outcomes when the initial rhythm is nonshockable.ObjectiveTo assess whether hypothermia at 33 °C increases survival and improves function when compared with controlled normothermia in unconscious adults resuscitated from OHCA with initial nonshockable rhythm.Data SourcesIndividual patient data meta-analysis of 2 multicenter, randomized clinical trials (Targeted Normothermia after Out-of-Hospital Cardiac Arrest [TTM2; NCT02908308] and HYPERION [NCT01994772]) with blinded outcome assessors. Unconscious patients with OHCA and an initial nonshockable rhythm were eligible for the final analysis.Study SelectionThe study cohorts had similar inclusion and exclusion criteria. Patients were randomized to hypothermia (target temperature 33 °C) or normothermia (target temperature 36.5 to 37.7 °C), according to different study protocols, for at least 24 hours. Additional analyses of mortality and unfavorable functional outcome were performed according to age, sex, initial rhythm, presence or absence of shock on admission, time to return of spontaneous circulation, lactate levels on admission, and the cardiac arrest hospital prognosis score.Data Extraction and SynthesisOnly patients who experienced OHCA and had a nonshockable rhythm with all causes of cardiac arrest were included. Variables from the 2 studies were available from the original data sets and pooled into a unique database and analyzed. Clinical outcomes were harmonized into a single file, which was checked for accuracy of numbers, distributions, and categories. The last day of follow-up from arrest was recorded for each patient. Adjustment for primary outcome and functional outcome was performed using age, gender, time to return of spontaneous circulation, and bystander cardiopulmonary resuscitation.Main Outcomes and MeasuresThe primary outcome was mortality at 3 months; secondary outcomes included unfavorable functional outcome at 3 to 6 months, defined as a Cerebral Performance Category score of 3 to 5.ResultsA total of 912 patients were included, 490 from the TTM2 trial and 422 from the HYPERION trial. Of those, 442 had been assigned to hypothermia (48.4%; mean age, 65.5 years; 287 males [64.9%]) and 470 to normothermia (51.6%; mean age, 65.6 years; 327 males [69.6%]); 571 patients had a first monitored rhythm of asystole (62.6%) and 503 a presumed noncardiac cause of arrest (55.2%). At 3 months, 354 of 442 patients in the hypothermia group (80.1%) and 386 of 470 patients in the normothermia group (82.1%) had died (relative risk [RR] with hypothermia, 1.04; 95% CI, 0.89-1.20; P = .63). On the last day of follow-up, 386 of 429 in the hypothermia group (90.0%) and 413 of 463 in the normothermia group (89.2%) had an unfavorable functional outcome (RR with hypothermia, 0.99; 95% CI, 0.87-1.15; P = .97). The association of hypothermia with death and functional outcome was consistent across the prespecified subgroups.Conclusions and RelevanceIn this individual patient data meta-analysis, including unconscious survivors from OHCA with an initial nonshockable rhythm, hypothermia at 33 °C did not significantly improve survival or functional outcome.

show abstract

The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials

Cited by 161 publications

References 81 publications

Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Efficacy and Safety of Flibanserin for the Treatment of Hypoactive Sexual Desire Disorder in Women

Hypothermia vs Normothermia in Patients With Cardiac Arrest and Nonshockable Rhythm

Contact Info

Product

Resources

About