An empirical analysis of eating disorders and anxiety disorders publications (1980-2000)—part II: Statistical hypothesis testing

Abstract

Objective: The current study compared the eating disorder literature and the anxiety disorder literature in terms of statistical hypothesis testing features in 1980, 1990, and 2000. Method: Computer literature searches were conducted using PubMed and PsychInfo databases to identify relevant eating disorder and anxiety disorder articles published at each of the three time points. A total of 456 articles were randomly selected, including 228 articles each from the fields of eating disorders and anxiety disorders. Within each field, one third (76) of the articles were selected from each of the three time points. Two raters, from a team of eight trained raters, were randomly assigned to independently rate each article in terms of 75 separate methodologic features. In the current article, we will emphasize the findings about hypothesis testing and statistical analysis. Disagreements in ratings were resolved via consensus. Ratings were tabulated separately by field across the three time points.Results: Few differences were observed between eating disorder and anxiety disorder publications in terms of statistical hypothesis testing features. Although increases were observed in both fields in a number of areas from 1980 to 2000, there remains a pervasive absence of many of the statistical hypothesis testing features recommended by the American Psychological Association Task Force on Statistical Inference. Conclusion:These results are discussed in terms of their implications for the fields of eating disorders and anxiety disorders, for researchers, for reviewers, and for professional journals and editorial boards. ª 2005 by Wiley Periodicals, Inc.

Tracing scientific reasoning in psychiatry: Reporting of statistical inference in abstracts of top journals 1975–2015

“…Both additional CIs and p values could have been found in full texts. A study that in fact evaluated full texts, however, found similar results: Crosby et al () analyzed 152 randomly selected eating and anxiety disorder research papers from 2000 and documented CIs in 11.8% (up from 0.6% in 152 papers from 1980) and p values in 85.6% (up from 62.6% in 1980). In comparison, we found no CIs in abstracts published in 1980–1984 and, among papers with any statistical inference, in 8% in 2000–2004.…”

“…Studies already exist on methodological problems in psychiatric articles, such as inconsistencies between test statistics and p values (Berle & Starcevic, ), deficiencies in the reporting and evaluation of blinding in psychiatric RCTs (Baethge, Assall, & Baldessarini, ), a low replication rate of psychiatric studies (Tajika, Ogawa, Takeshima, Hayasaka, & Furukawa, ), or plain statistical errors (Mcguigan, ). Although there are data on the presence of CIs in research on eating and anxiety disorders (Crosby et al, ), they are limited by the small number of articles analyzed and by the fact that the papers investigated were published between 1980 and 2000. The current and general pattern of describing statistical inference in psychiatry is unknown.…”

Current use of effect size or confidence interval analyses in clinical and biomedical research

“…Previous studies focused on eating and psychological disorders (Crosby et al, 2006 ), psychology (Faulkner et al, 2008 ; Fidler et al, 2005 ; Fritz et al, 2012a , b ), educational psychology (Osborne, 2008 ; Sun et al, 2010 ), and learning and education (Barry et al, 2016 ) are in consonance with the results presented in this research since it was observed that the effect size analysis is not routinely used in clinical and biomedical research. However, it is important to point out that the present study was performed without applying filters related to the magazine type, impact factor, or revision criteria, such as mandatory complementary statistical test report or peer reviews.…”

An empirical analysis of eating disorders and anxiety disorders publications (1980-2000)—part I: Research methods

“…Part II, which is reported in the accompanying article, 7 focuses on the process and procedures by which hypotheses are generated and statistically tested in scientific articles.…”