Tracing scientific reasoning in psychiatry: Reporting of statistical inference in abstracts of top journals 1975–2015

Baethge, Christopher; Deckert, Markus; Stang, Andreas

doi:10.1002/mpr.1735

Cited by 5 publications

(14 citation statements)

References 20 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proportion of papers found to have committed statistical errors was 45% in 1977–1978 and 40% in 1993. The reporting of statistical inference in abstracts of visible psychiatric journals has also been analysed (Baethge, Deckert, & Stang, ). The authors note that there is a shift to provide precise p values without invoking thresholds, such as p < .05.…”

Section: Introductionmentioning

confidence: 99%

Reporting of data analysis methods in psychiatric journals: Trends from 1996 to 2018

Nieminen

Kaur

2019

Int J Methods Psych Res

View full text Add to dashboard Cite

Objectives The article aims to evaluate how study designs and data analysis methods in psychiatric studies have changed over the last 22 years. Methods This study involved a total of 320 papers published in 1996 and 2018 in the American Journal of Psychiatry, Acta Psychiatrica Scandinavica, British Journal of Psychiatry, and JAMA Psychiatry. We manually reviewed the articles to determine the way in which they reported the study characteristics and the methods applied in data analysis. Results The statistical intensity in psychiatric journals has changed over the past 20 years. Traditional methods of testing statistical significance were widely used both in 1996 and in 2018. In 2018, there was an increase in reporting more complex methods, such as multivariable regression models, multilevel modelling, and intracluster correlation methods. However, computationally complex data mining or machine learning procedures were not adopted by psychiatric researchers. Conclusion The increase in statistical intensity in the literature suggests that readers of prominent psychiatric journals must possess a substantial level of statistical expertise if they wish to critically evaluate the findings published in these journals. It is essential to include an awareness of this substantial change in data analysis methods in psychiatric undergraduate and postgraduate education.

show abstract

Section: Introductionmentioning

confidence: 99%

Reporting of data analysis methods in psychiatric journals: Trends from 1996 to 2018

Nieminen

Kaur

2019

Int J Methods Psych Res

View full text Add to dashboard Cite

show abstract

“…The ambiguous usage of "significance", "significantly, "and "significant "without qualifying it as substantive or statistical was present in abstract and full text. The same results were reached in former reviews [18,19]. Moreover, the use of the term "significance" might have changed over time; however, inferring the intended meaning from the publication, itself can be complicated and it might require contacting authors, but even that is difficult, if not impossible.…”

Section: Discussionmentioning

confidence: 53%

“…These developments lead to hypotheses that NHT might be losing popularity, and that ST and CI reporting might be increasing, in recent years. A review of abstracts in epidemiological, medical, and psychiatry journals did find an increasing trend in CI reporting, while NHT reporting was the dominant statistical inference in abstracts containing statistical inference [18,19]. However, to our knowledge data on the time trend in reporting statistical inferences in abstracts published in pharmacology journals are missing.…”

Section: Introductionmentioning

confidence: 72%

“…Systematic reviews of other types of journals from 1975 through 2014 showed similar time trends in the use of CIs [18]. In contrast, uptake of CIs was slower by psychology journals over almost the same period [19]. This shows the variation in acceptance of CIs by different medical journals and scientific communities.…”

Section: Discussionmentioning

confidence: 94%

“…Therefore, we extracted the data up to 2016. Because previous studies revealed that abstracts regularly appeared not earlier than in the mid-70s, the observation periods starts in 1976 [18,19]. Raw data regarding the publication year 1976 and 2016 is provided in online supplement (Table S5 and S6).…”

Section: Acknowledgementsmentioning

confidence: 99%

See 2 more Smart Citations

Statistical inference in abstracts of 3 influential clinical pharmacology journals analysed using a text‐mining algorithm

Amiri

Deckert

Michel

et al. 2021

Brit J Clinical Pharma

Self Cite

View full text Add to dashboard Cite

To describe the trend in the prevalence of statistical inference in three influential clinical pharmacology journals Methods: We applied a computer-based algorithm to abstracts of three clinical pharmacology journals published in 1976 to 2016 to identify statistical inference and its subtypes. Furthermore, we manually reviewed a random sample of 300 articles to access algorithm's performance in finding statistical inference in abstracts and as a screening tool for presence and absence of statistical inference in full text. Result:The algorithm identified 59% (13,375/22,516 [mid p 95% CI, 59%-60%]) article abstracts with statistical inference. The percentage of abstracts with statistical inference was similar in 1976 and 2016, 48% (179/377 [mid p 95%CI, 42%-52%]) versus 49% (386/791 [mid p 95%CI, 45%-52%]). Statistical reporting pattern varied among journals. Among abstracts containing any statistical inference in the publications from 1976 to 2016 null-hypothesis significance testing was the most prevalent reported statistical inference. The algorithm had high sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for finding statistical inferences in abstract. While PPV for predicting the statistical inference in full text (including abstract, text, tables and figures) was high, NPV was low. Conclusion:Despite journal's editorials and statistical associations' guidelines, most authors focused on testing rather than estimation. In future, a better statistical reporting might be ensured by improving the statistical knowledge of authors and an addition of statistical guides to journals' instruction to authors to the extent that editors would like their statistical inference preferences to be incorporated into submitted manuscripts.

show abstract

Reporting of Statistical Inference in Abstracts of Major Cancer Journals, 1990 to 2020

Stang

Schmidt

2022

JAMA Netw Open

Self Cite

View full text Add to dashboard Cite

show abstract

Tracing scientific reasoning in psychiatry: Reporting of statistical inference in abstracts of top journals 1975–2015

Cited by 5 publications

References 20 publications

Reporting of data analysis methods in psychiatric journals: Trends from 1996 to 2018

Reporting of data analysis methods in psychiatric journals: Trends from 1996 to 2018

Statistical inference in abstracts of 3 influential clinical pharmacology journals analysed using a text‐mining algorithm

Reporting of Statistical Inference in Abstracts of Major Cancer Journals, 1990 to 2020

Contact Info

Product

Resources

About