Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

Shinohara, Kiyomi; Aoki, Takuya; So, Ryuhei; Tsujimoto, Yasushi; Suganuma, Aya M; Kise, Morito; Furukawa, Toshi A

doi:10.1136/bmjopen-2017-018355

Cited by 10 publications

(9 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While definitive conclusions about the effect of spin on real-life clinical practice are difficult to ascertain based off one trial of spin, the results of the Boutron trial suggest that there is significant potential for misinterpretation of results when spin is introduced to article abstracts. Conversely, another 2017 RCT of 622 clinicians found that when the primary outcome of a trial is clear in the abstract, the presence of spin or overstatement of nonsignificant results does not change the perception of the efficacy of the treatment 21. In fact, clinicians found trials without spin in the abstract to be more valid than those with overstated results.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of spin in abstracts of papers in psychiatry and psychology journals

Jellison

Roberts

Bowers

et al. 2019

BMJ EBM

View full text Add to dashboard Cite

We have identified ‘spin’ in abstracts of randomised controlled trials (RCTs) with nonsignificant primary endpoints in psychiatry and psychology journals. This is a cross-sectional review of clinical trials with nonsignificant primary endpoints published in psychiatry and psychology journals from January 2012 to December 2017. The main outcome was the frequency and manifestation of spin in the abstracts. We define spin as the ‘use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results’. We have also assessed the relationship between industry funding and spin. Of the 486 RCTs examined, 116 were included in our analysis of spin. Spin was identified in 56% (n=65) of those included. Spin was found in 2 (2%) titles, 24 (21%) abstract results sections and 57 (49.1%) abstract conclusion sections. Evidence of spin was simultaneously identified in both results and conclusions sections in 15% of RCTs (n=17). Twelve articles reported industry funding (10%). Industry funding was not associated with increased odds of spin in the abstract (unadjusted OR: 1.0; 95% CI: 0.3 to 3.2). We found no relationship between industry funding and spin in abstracts. These findings raise concerns about the effects spin may have on clinicians. Further steps could be taken to address spin, including inviting reviewers to comment on the presence of spin and updating Consolidated Standards of Reporting Trials guidelines to contain language discouraging spin.

show abstract

Section: Discussionmentioning

confidence: 99%

Evaluation of spin in abstracts of papers in psychiatry and psychology journals

Jellison

Roberts

Bowers

et al. 2019

BMJ EBM

View full text Add to dashboard Cite

show abstract

“…The first researcher (RS) developed all the Web systems for the DOCTOR study. Details of the methods and results of the DOCTOR study are reported elsewhere [ 19 ].…”

Section: Methodsmentioning

confidence: 99%

“…We hereby report the results from the first phase. The English version of the invitation email is included in the supplementary appendix reported elsewhere [ 19 ].…”

Section: Methodsmentioning

confidence: 99%

Effect of Recruitment Methods on Response Rate in a Web-Based Study for Primary Care Physicians: Factorial Randomized Controlled Trial

So¹,

Shinohara²,

Aoki³

et al. 2018

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

BackgroundLow participation rates are one of the most serious disadvantages of Web-based studies. It is necessary to develop effective strategies to improve participation rates to obtain sufficient data.ObjectiveThe objective of this trial was to investigate the effect of emphasizing the incentive in the subject line of the invitation email and the day of the week of sending the invitation email on the participation rate in a Web-based trial.MethodsWe conducted a 2×2 factorial design randomized controlled trial. We contacted 2000 primary care physicians from members of the Japan Primary Care Association in January 2017 and randomly allocated them to 1 of 4 combinations of 2 subject lines (presence or absence of an emphasis on a lottery for an Amazon gift card worth 3000 yen or approximately US $30) and 2 delivery days (sending the invitation email on Tuesday or Friday). The primary outcome was the response rate defined as the number of participants answering the first page of the questionnaire divided by the number of invitation emails delivered. All outcomes were collected between January 17, 2017, and February 8, 2017.ResultsWe analyzed data from 1943 out of 2000 participants after excluding those whose email addresses were invalid. The overall response rate was 6.3% (123/1943). There was no significant difference in the response rates between the 2 groups regarding incentive in the subject line: the risk ratio was 1.12 (95% CI 0.80 to 1.58) and the risk difference was 0.7% (95% CI –1.5% to 2.9%). Similarly, there was no significant difference in the response rates between the 2 groups regarding sending the email on Tuesday or Friday: the risk ratio was 0.98 (95% CI 0.70 to 1.38) and the risk difference was –0.1% (95% CI –2.3% to 2.1%).ConclusionsNeither emphasizing the incentive in the subject line of the invitation email nor varying the day of the week the invitation email was sent led to a meaningful increase in response rates in a Web-based trial with primary care physicians.Trial RegistrationUniversity Hospital Medical Information Network Clinical Trials Registry UMIN000025317; https://upload.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000029121 (Archived by WebCite at http://www.webcitation. org/6wOo1jl9t)

show abstract

“…(DOCTOR) study is a double-blind RCT that was conducted between January and February 2017. The methods of this study and its primary results have been published elsewhere [9] and are only briefly described here.…”

Section: Methodsmentioning

confidence: 99%

Physician characteristics associated with proper assessment of overstated conclusions in research abstracts: A secondary analysis of a randomized controlled trial

et al. 2019

Self Cite

View full text Add to dashboard Cite

ObjectivesLittle is known about the physician characteristics associated with appraisal skills of research evidence, especially the assessment of the validity of study methodology. This study aims to explore physician characteristics associated with proper assessment of overstated conclusions in research abstracts.DesignA secondary analysis of a randomized controlled trial.Setting and participantsWe recruited 567 volunteers from the Japan Primary Care Association.MethodsParticipants were randomly assigned to read the abstract of a research paper, with or without an overstatement, and to rate its validity. Our primary outcome was proper assessment of the validity of its conclusions. We investigated the association of physician characteristics and proper assessment using logistic regression models and evaluated the interaction between the associated characteristics and overstatement.ResultsWe found significant associations between proper assessment and post-graduate year (odds ratio [OR] = 0.67, 95% confidence interval [CI] 0.49 to 0.91, for every 10-year increase) and research experience as a primary investigator (PI; OR = 2.97, 95% CI 1.65 to 5.34). Post-graduate year and PI had significant interaction with overstatement (P = 0.015 and < 0.001, respectively). Among participants who read abstracts without an overstatement, post-graduate year was not associated with proper assessment (OR = 1.04, 95% CI 0.82 to 1.33), and PI experience was associated with lower scores of the validity (OR = 0.58, 95% CI 0.35 to 0.96).ConclusionPhysicians who have been in practice longer should be trained in distinguishing overstatements in abstract conclusions. Physicians with research experience might be informed that they tend to rate the validity of research lower regardless of the presence or absence of overstatements.Trial registrationUMIN000026269.

show abstract

Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

Cited by 10 publications

References 28 publications

Evaluation of spin in abstracts of papers in psychiatry and psychology journals

Evaluation of spin in abstracts of papers in psychiatry and psychology journals

Effect of Recruitment Methods on Response Rate in a Web-Based Study for Primary Care Physicians: Factorial Randomized Controlled Trial

Physician characteristics associated with proper assessment of overstated conclusions in research abstracts: A secondary analysis of a randomized controlled trial

Contact Info

Product

Resources

About