Trends in the Use of Promotional Language (Hype) in National Institutes of Health Funding Opportunity Announcements, 1992-2020

Millar, Neil; Batalo, Bojan; Budgell, Brian

doi:10.1001/jamanetworkopen.2022.43221

Cited by 6 publications

(18 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While effective communication has long been central to the conduct of science, this report by Millar and colleagues highlights how scientists convey the quality—in this case the anticipated quality—of their work. Not surprisingly, scientists believe that their work is and will be of high quality.…”

mentioning

confidence: 75%

“…Millar and colleagues describe an analysis of the use of promotional language (“hype”) in abstracts of National Institutes of Health (NIH)–funded research projects. They extracted more than 900 000 abstracts from the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive and measured the frequency of certain terms from 1985 to 2020.…”

mentioning

confidence: 99%

“…The report by Millar and colleagues raises the question as to whether there are alternatives to promotional adjectives to convey the novelty and rigor—or lack of rigor—of scientific proposals or reports. Bibliometric methods exist to distinguish science that is disruptive as opposed to developmental or incremental.…”

mentioning

confidence: 99%

“…Scientists may use promotional adjectives to describe their work—both work they propose and work they report—but, as Millar and colleagues imply, we as a scientific community need to transcend subjective language and appeals to emotion. Many scientific leaders and funders, including the NIH, are undertaking efforts to improve the rigor of scientific design, conduct, and reporting.…”

mentioning

confidence: 99%

“…The report by Millar and colleagues itself reflects a form of transparency. The authors took advantage of the transparency of the NIH RePORTER archive, and they have decided to share their data for others to use.…”

mentioning

confidence: 99%

See 4 more Smart Citations

From Hype to High-Quality Research

Lauer

2022

JAMA Netw Open

View full text Add to dashboard Cite

Millar and colleagues 1 describe an analysis of the use of promotional language ("hype") in abstracts of National Institutes of Health (NIH)-funded research projects. They extracted more than 900 000 abstracts from the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive and measured the frequency of certain terms from 1985 to 2020. They found that the use of most promotional adjectives-such as novel, critical, and innovative-gradually and continuously increased, while the use of others-such as scalable and transformative-was essentially nonexistent in 1985 but later became much more common beginning in the late 2000s. The authors

show abstract

mentioning

confidence: 75%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

From Hype to High-Quality Research

Lauer

2022

JAMA Netw Open

View full text Add to dashboard Cite

show abstract

Promotional Language (Hype) in Abstracts of Publications of National Institutes of Health–Funded Research, 1985-2020

Millar,

Batalo,

Budgell

2023

JAMA Netw Open

Self Cite

View full text Add to dashboard Cite

ImportanceInvestigators applying for National Institutes of Health (NIH) funding increasingly use promotional language (or hype) that has the potential to undermine objective evaluation. Whether or not the same investigators use hype in subsequent research reports has yet to be investigated.ObjectiveTo assess changes in the use of hype in journal abstracts reporting research funded by the NIH and to compare those trends with previously reported trends in the associated NIH funding applications.Design, Setting, and ParticipantsThis cross-sectional study assessed trends (from 1985 to 2020) in the use of promotional adjectives in abstracts of journal articles reporting NIH-funded research, and then compared those trends with previously reported trends for the associated NIH funding applications. Articles included in analyses had abstracts available in PubMed.Main Outcomes and MeasuresAbsolute change for the 139 adjective forms that have previously been identified as representing hype in NIH funding applications was measured as the difference in frequency between 1985 and 2020. Relative change was measured as the percentage change in frequency in 2020 relative to 1985, or the first year of occurrence. Consistency of change was measured by the rank order correlation (Kendall τ). Concordance between longitudinal trends in the journal abstracts and NIH funding applications was measured by the rank-order cross-correlation.ResultsIn a total of 2 394 480 journal abstracts, all 139 adjective forms were identified in 2 793 592 total occurrences. Among these adjectives, 133 increased in absolute frequency by 5335 words per million (wpm), with a mean (SD) relative increase of 1404% (2371%). The largest absolute increases were for novel (524 wpm), important (414 wpm), and key (378 wpm). The largest relative increases were for scalable (22 wpm [19 964%]), unmet (23 wpm [12 126%]), and tailored (40 wpm [8169%]). The mean (SD) correlation for all adjectives was 0.70 (0.30) with 95 adjectives showing a strong positive correlation (τ &gt; 0.7; P &lt; .001), 24 a moderate positive correlation (0.5 &lt; τ &lt; 0.7; P &lt; .001), and 3 a moderate negative correlation (−0.5 &lt; τ &lt; −0.7; P &lt; .001). The mean (SD) cross-correlation was 0.64 (0.19) with 61 of the 139 adjectives showing a strong positive cross-correlations (τ &gt; 0.7; P &lt; .001), 53 a moderate positive cross-correlations (0.5 &lt; τ &lt; 0.7; P &lt; .001), and 3 a moderate negative cross-correlation (−0.7 &lt; τ &lt; −0.5; P &lt; .001).Conclusions and RelevanceIn this analysis of journal abstracts reporting NIH-funded research from 1985 to 2020, levels of promotional language were found to be increasing and trends were closely associated with previously reported trends in the related NIH funding applications. This suggests that increasing levels of salesmanship may in part be a downstream effect of salesmanship infused during earlier stages of the research cascade.

show abstract

Hype, the Responsibility of Authors and Editors, and the Subjective Interpretation of Evidence

Bauchner

2023

JAMA Netw Open

View full text Add to dashboard Cite

Trends in the Use of Promotional Language (Hype) in National Institutes of Health Funding Opportunity Announcements, 1992-2020

Abstract: This cross-sectional study examines changes from 1992 to 2020 in the use of promotional language in National Institutes of Health (NIH) funding opportunity announcements in comparison with trends reported in NIH grant applications.

Cited by 6 publications

References 4 publications

From Hype to High-Quality Research

From Hype to High-Quality Research

Promotional Language (Hype) in Abstracts of Publications of National Institutes of Health–Funded Research, 1985-2020

Hype, the Responsibility of Authors and Editors, and the Subjective Interpretation of Evidence

Contact Info

Product

Resources

About