Reporting of data analysis methods in psychiatric journals: Trends from 1996 to 2018

Abstract

Objectives The article aims to evaluate how study designs and data analysis methods in psychiatric studies have changed over the last 22 years. Methods This study involved a total of 320 papers published in 1996 and 2018 in the American Journal of Psychiatry, Acta Psychiatrica Scandinavica, British Journal of Psychiatry, and JAMA Psychiatry. We manually reviewed the articles to determine the way in which they reported the study characteristics and the methods applied in data analysis. Results The statistical intensity in psychiatric journals has changed over the past 20 years. Traditional methods of testing statistical significance were widely used both in 1996 and in 2018. In 2018, there was an increase in reporting more complex methods, such as multivariable regression models, multilevel modelling, and intracluster correlation methods. However, computationally complex data mining or machine learning procedures were not adopted by psychiatric researchers. Conclusion The increase in statistical intensity in the literature suggests that readers of prominent psychiatric journals must possess a substantial level of statistical expertise if they wish to critically evaluate the findings published in these journals. It is essential to include an awareness of this substantial change in data analysis methods in psychiatric undergraduate and postgraduate education.

Use of Data Analysis Methods in Dental Publications: Is There Evidence of a Methodological Change?

“…The availability of statistical software packages has greatly facilitated extensive data analysis and increased the quantity and complexity of using statistical methods. Our findings are in line with previous findings from other medical sub-fields, where clinical researchers have not approved data mining and computationally intensive machine learning methods [25,34,43]. So far, machine learning algorithms that have been developed for prediction purposes have not been useful tools in clinical medicine when the purpose is to understand factors that affect developing diseases or estimate the effectiveness of new treatment methods [44].…”

“…Over 33% of the evaluated articles analyzed data that included at least 300 study subjects in 2017. This proportion is still low when compared to articles published in visible psychiatric (63%) or respiratory (50%) journals [25,34]. The previously reported alarming phenomenon of decreasing sample sizes in dentistry has hopefully changed to a trend of greater precision and power [8].…”

“…We have measured, with slight modifications, the same variables as in our previous study from psychiatric articles [25]. First, the number of authors and international collaboration was recorded.…”

“…Several multilevel data analysis methods have been proposed for analyzing clustered data that do not ignore the clustering structure of surface and teeth [22,23]. In addition, various analysis methods of agreement and reliability have been implemented in dentistry to measure the extent of agreement among raters, or to validate the questionnaires or diagnostic tests [24,25]. The broad introduction of new data analysis methods that are compatible with the rapid expansion in computing capability has reached also medical publications [26][27][28].…”

Ten Points for High-Quality Statistical Reporting and Data Presentation

“…Tables and Figures? The medical literature shows a strong tendency to accentuate significance testing, specifically "statistically significant" outcomes. Most papers published in medical journals contain tables and figures reporting p-values [16,20,21]. Finding statistically significant or nonsignificant results depends on the sample size [22,23].…”