Long-Term Trends (1980–2017) in the N-Pact Factor of Journals in Personality Psychology and Individual Differences Research

Abstract

Abstract. Recent metascience investigations of the N-pact factor (NF; median sample size of studies published in a journal) have revealed NFs of merely about 100 in fields like social, sport, and exercise psychology. Journal NF has also been shown to correlate negatively with journal impact factors (JIF), implying that smaller studies appear in more prestigious journals. In this first long-term and largest NF analysis to date (3,699 articles coded), annual NFs of two personality psychology journals were tracked over 38 years since their inception in 1980. Overall NF was about 190, gradually increased over time, and within-journal NF-JIF correlations were positive. Online samples and articles featuring supplemental files presented larger NFs, whereas those involving student samples had smaller ones. Sample size distributions showed multimodality, and surplusses of even-numbered sample sizes and of those just beyond 100 were evident. An NF statement, accompanying authors’ submitted papers, is suggested.

Replicability, Robustness, and Reproducibility in Psychological Science

“…Multiple assessments of statistical power in psychology suggest that statistical power has been consistently low and has not substantially increased from 1955 to 2014 (Cohen, 1962;Rossi, 1990;Sedlmeier & Gigerenzer, 1992;Smaldino & McElreath, 2016;Stanley et al, 2018). Metaresearchers have characterized the median sample sizes of studies sampled from psychology journals from 1977 to 2018 (Fraley & Vazire, 2014;Kossmeier et al, 2019;Marszalek et al, 2011;Reardon et al, 2019;Schweizer & Furley, 2016). Our synthesis of these data (available at osf.io/nb2fv) suggests that sample sizes in almost all research areas have been largely stable.…”

“…There is not clear evidence that longitudinal patterns of statistical power and sample sizes over time differ as a function of research area. Sample sizes do differ on average as a function of research contexts; sample sizes tend to be relatively lower in highly resourceintensive research areas (e.g., neuroscience, Button et al, 2013), in experimental designs relative to correlational designs (Kossmeier et al, 2019;Schweizer & Furley, 2016), and, likely, in studies that aren't and can't be conducted online (S. F. Anderson et al, 2017). Apparent differences in longitudinal patterns, for example, in personality psychology relative to sports and exercise psychology, may be due to differences in the complexity and design of personality research rather than to a collective change in research practices among personality psychologists.…”

Replicability, Robustness, and Reproducibility in Psychological Science

“…Multiple assessments of statistical power in psychology suggest that statistical power has been consistently low and has not substantially increased from 1955 to 2014 Rossi, 1990;Stanley et al, 2018). Meta-researchers have characterized the median sample sizes of studies sampled from psychology journals from 1977 to 2018 (Fraley & Vazire, 2014;Kossmeier et al, 2019;Marszalek et al, 2011;Reardon et al, 2019;Schweizer & Furley, 2016). Our synthesis of these data (available at osf.io/nb2fv) suggests that sample sizes in almost all research areas have been largely stable.…”

“…However, sample size is merely a proxy for power. If people are using simpler designs with similar Ns (as suggested by Kossmeier et al, 2019;Schweizer & Furley, 2016), then power may be increasing despite the stability in sample size.…”

“…There is not clear evidence that longitudinal patterns of statistical power and sample sizes over time differ as a function of research area. Sample sizes do differ on average as a function of research contexts; sample sizes tend to be relatively lower in highly resourceintensive research areas (e.g., neuroscience, , in experimental designs relative to correlational designs (Kossmeier et al, 2019;Schweizer & Furley, 2016), and, likely, in studies that aren't and can't be conducted online (Anderson et al, 2017). Apparent differences in longitudinal patterns, for example, in personality psychology relative to sports and exercise psychology, may be due to differences in the complexity and design of personality research rather than to a collective change in research practices among personality psychologists.…”