Back to Basics: The Importance of Measurement Properties in Biological Psychiatry

Abstract

Biological psychiatry is a major funding priority for organizations that fund mental health research (e.g., National Institutes of Health). Despite this, some have argued that the field has fallen short of its considerable promise to meaningfully impact the classification, diagnosis, and treatment of psychopathology. This may be attributable in part to a paucity of research about key measurement properties (“physiometrics”) of biological variables as they are commonly used in biological psychiatry research. Specifically, study designs informed by physiometrics are more likely to be replicable, avoid measurement concerns that drive down effect sizes, and maximize efficiency in terms of time, money, and the number of analyses conducted. This review describes four key physiometrics (dimensionality, internal consistency, temporal stability, and temporal specificity), illustrates how lack of understanding about these characteristics imposes meaningful limitations on research, and reviews examples of physiometric studies featuring a variety of popular biological variables to illustrate how this research can be done and substantive conclusions about the variables of interest.

Hierarchical Inflammatory Phenotypes of Depression: A Novel Approach across Five Independent Samples and 27,730 Adults

“…However, given that this bias only was seen in two of the nine symptoms and that effect sizes were small, this is likely not a hugely influential issue for CRP and the PHQ-9. It is worth note that, while small, the effect Hierarchical Inflammatory Phenotypes of Depression Moriarity et al 12 sizes observed in this study are larger than the average effect sizes found in a recent metaanalysis (52), highlighting the possibility that structural equation modeling's ability to correct for measurement error might deattenuate downward-biased effect sizes resulting from unreliable measures (53,54). Further, if CRP is a truly a unique predictor of both latent depression (i.e., the variance shared among depression symptoms) and individual symptoms, larger effect sizes might be the result of selecting a model that more closely matches the naturally-occurring relations (resulting in less error).…”

Robust group- but limited individual-level (longitudinal) reliability and insights into cross-phases response prediction of conditioned fear

“…Our work follows recent calls for increasing attention to measurement reliability as a crucial determinant of statistical power in psychology and the neurosciences (Fröhner, Teckentrup, Smolka, & Kroemer, 2019; Hedge, Powell, & Sumner, 2018; Moriarity & Alloy, 2021; Zuo, Xu, & Milham, 2019). Measurement reliability is of key importance when addressing clinically-relevant questions involving individual-level predictions such as predicting treatment success from extinction (e.g., Lange et al, 2020; Scheveneels, Boddez, Vervliet, & Hermans, 2016) or biomarkers (Cano-Catala et al, 2021).…”

“…However, a remarkable amount of predictions were non-significant - which was particularly true for CS discrimination in SCRs and BOLD. This may be explained by difference scores (i.e., CS+ minus CS-) being generally less reliable (Lynam, Hoyle, & Newman, 2006) due to a subtraction of meaningful variance (Moriarity & Alloy, 2021) particularly in highly-correlated predictors (Thomas & Zumbo, 2012).…”

“…Such work on measurement questions can (often) be done cost and resource effective in existing or openly available data - which, however, requires cross-lab data sharing and data management homogenization plans. Devoting resources and funds to measurement optimization is a valuable investment into the prospect of this field contributing to improved mental health (Moriarity & Alloy, 2021) and to resume the path to successful translation from neuroscience discoveries into clinical applications.…”

“…The crucial role of adequate statistical power in neuroscience (Button et al, 2013) has gained momentum in the past years and resulted in calls for and efforts to increase sample sizes (Moriarity & Alloy, 2021;Szucs & Ioannidis, 2020). Measurement reliability, however, has only recently gained attention as a crucial determinant of the maximally observable effect size (Fröhner, Teckentrup, Smolka, & Kroemer, 2019;Hedge, Powell, & Sumner, 2018;Zuo, Xu, & Milham, 2019).…”