Overstatements in abstract conclusions claiming effectiveness of interventions in psychiatry: A meta-epidemiological investigation

Shinohara, Kiyomi; Suganuma, Aya M; Takeshima, Nozomi; Hayasaka, Yu; Furukawa, Toshi A

doi:10.1371/journal.pone.0184786

Cited by 14 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, this study used the statements in the scientific publications as a baseline, but many peer reviewed publications already contain exaggerations ( Haber et al ., 2018; Lazarus et al ., 2015; Mathieu et al ., 2012; Shinohara et al ., 2017). This means that the actual level of exaggeration in the media may be higher than reported and this offers another challenge for journalists, who cannot take the scientific articles at face value.…”

Section: Discussionmentioning

confidence: 99%

Expert quotes and exaggeration in health news: a retrospective quantitative content analysis

et al. 2019

View full text Add to dashboard Cite

BackgroundThis research is an investigation into the role of expert quotes in health news, specifically whether news articles containing a quote from an independent expert are less often exaggerated than articles without such a quote.MethodsRetrospective quantitative content analysis of journal articles, press releases, and associated news articles was performed. The investigated sample are press releases on peer-reviewed health research and the associated research articles and news stories. Our sample consisted of 462 press releases and 668 news articles from the UK (2011) and 129 press releases and 185 news articles from The Netherlands (2015). We hand-coded all journal articles, press releases and news articles for correlational claims, using a well-tested codebook. The main outcome measures are types of sources that were quoted and exaggeration of correlational claims. We used counts, 2x2 tables and odds ratios to assess the relationship between presence of quotes and exaggeration of the causal claim.ResultsOverall, 99.1% of the UK press releases and 84.5% of the Dutch press releases contain at least one quote. For the associated news articles these percentages are: 88.6% in the UK and 69.7% in the Netherlands. Authors of the study are most often quoted and only 7.5% of UK and 7.0% of Dutch news articles contained a new quote by an expert source, i.e. one not provided by the press release. The relative odds that an article without an external expert quote contains an exaggeration of causality is 2.6.ConclusionsThe number of articles containing a quote from an independent expert is low, but articles that cite an external expert do contain less exaggeration.

show abstract

Section: Discussionmentioning

confidence: 99%

Expert quotes and exaggeration in health news: a retrospective quantitative content analysis

et al. 2019

View full text Add to dashboard Cite

show abstract

“…After registering baseline characteristics, participants were randomly allocated to a research abstract with or without overstatements. An overstatement was defined as ‘inconsistency between the results of primary outcomes in full-text and those deduced from the abstract conclusion’ [7]. We selected four abstracts from PubMed as prototypical examples of overstatement.…”

Section: Methodsmentioning

confidence: 99%

“…Boutron et al suggested that more than half of the abstracts or main texts of RCT reports with no significant results for any primary outcomes showed inconsistency between the results and the conclusions in their abstracts, and called such inconsistency a “spin” [6]. Our previous study also showed that approximately one third of psychiatry studies claiming effectiveness in their abstract conclusions were exaggerated in comparison with the full text results [7]. Such inappropriate reporting in the abstract, that may be called spin or overstatement, can influence physicians’ interpretation if their critical appraisal skills are limited [6, 7].…”

Section: Introductionmentioning

confidence: 99%

“…Our previous study also showed that approximately one third of psychiatry studies claiming effectiveness in their abstract conclusions were exaggerated in comparison with the full text results [7]. Such inappropriate reporting in the abstract, that may be called spin or overstatement, can influence physicians’ interpretation if their critical appraisal skills are limited [6, 7]. Boutron et al conducted an RCT and found that the researchers who read an abstract with a spin were more likely to think that the intervention was beneficial for the patients than those who read an abstract without a spin [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Physician characteristics associated with proper assessment of overstated conclusions in research abstracts: A secondary analysis of a randomized controlled trial

et al. 2019

Self Cite

View full text Add to dashboard Cite

ObjectivesLittle is known about the physician characteristics associated with appraisal skills of research evidence, especially the assessment of the validity of study methodology. This study aims to explore physician characteristics associated with proper assessment of overstated conclusions in research abstracts.DesignA secondary analysis of a randomized controlled trial.Setting and participantsWe recruited 567 volunteers from the Japan Primary Care Association.MethodsParticipants were randomly assigned to read the abstract of a research paper, with or without an overstatement, and to rate its validity. Our primary outcome was proper assessment of the validity of its conclusions. We investigated the association of physician characteristics and proper assessment using logistic regression models and evaluated the interaction between the associated characteristics and overstatement.ResultsWe found significant associations between proper assessment and post-graduate year (odds ratio [OR] = 0.67, 95% confidence interval [CI] 0.49 to 0.91, for every 10-year increase) and research experience as a primary investigator (PI; OR = 2.97, 95% CI 1.65 to 5.34). Post-graduate year and PI had significant interaction with overstatement (P = 0.015 and < 0.001, respectively). Among participants who read abstracts without an overstatement, post-graduate year was not associated with proper assessment (OR = 1.04, 95% CI 0.82 to 1.33), and PI experience was associated with lower scores of the validity (OR = 0.58, 95% CI 0.35 to 0.96).ConclusionPhysicians who have been in practice longer should be trained in distinguishing overstatements in abstract conclusions. Physicians with research experience might be informed that they tend to rate the validity of research lower regardless of the presence or absence of overstatements.Trial registrationUMIN000026269.

show abstract

“… 5 Previous studies also found that 58% of RCTs with non-significant results, 5 and 70% of non-randomised studies 7 had spin. Subsequent studies reported that spin, misleading information or overstatements were common in various subspecialties, such as rheumatology, 8 psychiatry, 9 wound care, 10 surgery 11 12 and oncology. 13–15 …”

Section: Introductionmentioning

confidence: 99%

Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

Shinohara

Aoki

et al. 2017

BMJ Open

Self Cite

View full text Add to dashboard Cite

ObjectivesTo investigate whether overstatements in abstract conclusions influence primary care physicians’ evaluations when they read reports of randomised controlled trials (RCTs)DesignRCT setting: This study was a parallel-group randomised controlled survey, conducted online while masking the study hypothesis.ParticipantsVolunteers were recruited from members of the Japan Primary Care Association in January 2017. We sent email invitations to 7040 primary care physicians. Among the 787 individuals who accessed the website, 622 were eligible and automatically randomised into ‘without overstatement’ (n=307) and ‘with overstatement’ (n=315) groups.InterventionsWe selected five abstracts from published RCTs with at least one non-significant primary outcome and overstatement in the abstract conclusion. To construct a version without overstatement, we rewrote the conclusion sections. The methods and results sections were standardised to provide the necessary information of primary outcome information when it was missing in the original abstract. Participants were randomly assigned to read an abstract either with or without overstatements and asked to evaluate the benefit of the intervention.Outcome measuresThe primary outcome was the participants’ evaluation of the benefit of the intervention discussed in the abstract, on a scale from 0 to 10. A secondary outcome was the validity of the conclusion.ResultsThere was no significant difference between the groups with respect to their evaluation of the benefit of the intervention (mean difference: 0.07, 95% CI −0.28 to 0.42, p=0.69). Participants in the ‘without’ group considered the study conclusion to be more valid than those in the ‘with’ group (mean difference: 0.97, 95% CI 0.59 to 1.36, P<0.001).ConclusionThe overstatements in abstract conclusions did not significantly influence the primary care physicians’ evaluations of the intervention effect when necessary information about the primary outcomes was distinctly reported.Trial registration number UMIN000025317; Pre-results.

show abstract

Overstatements in abstract conclusions claiming effectiveness of interventions in psychiatry: A meta-epidemiological investigation

Cited by 14 publications

References 30 publications

Expert quotes and exaggeration in health news: a retrospective quantitative content analysis

Expert quotes and exaggeration in health news: a retrospective quantitative content analysis

Physician characteristics associated with proper assessment of overstated conclusions in research abstracts: A secondary analysis of a randomized controlled trial

Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

Contact Info

Product

Resources

About