Longitudinal measurement invariance of the Behavioral Health Measure in a clinical sample.

Abstract

The practice of routine outcome monitoring (ROM) has grown in popularity and become a fixture in feedback-supported clinical practice and research. However, if the interpretation of an ROM measure changes over time, treatment outcome scores may be inaccurate and produce erroneous or misguided interpretations of client progress and therapist efficacy. The current study examined whether factorial invariance held when using the Behavioral Health Measure (BHM-20) longitudinally in a clinical sample (n ϭ 12,467). Using multidimensional item response theory-based models for the investigation of the BHM-20 factor structure, at a single time point and then longitudinally. Based on the original factor structure of the BHM-20 a unidimensional model, a three-factor orthogonal model, and a three-factor correlated model were fit to the data, indicating poor model fit with the proposed three-factor or unidimensional models. Next, using exploratory factor analysis and subsequent multidimensional item response theory procedures, a new 4-factor (General Distress, Life Functioning, Anxiety, and Alcohol/ Drug Use) model was proposed with improved model-fit statistics. Finally, when testing the longitudinal invariance of the BHM-17 over 10 sessions of treatment, it was found to be fully consistent. The current study proposes the use of a 17-item, 4-factor model for a new understanding of the BHM-17. Implications for use in ROM and limitations are discussed. Public Significance StatementThe current study suggests that instruments used in ROM, specifically the BHM-20, may not be accurate in measuring client outcomes longitudinally. This is problematic because it means that client feedback data collected in session about their experience in therapy is not accurate. The current study offers a refined version of the BHM-20 called the BHM-17, which can be used longitudinally with improved accuracy.

Validation of the Cooper–Norcross Inventory of Preferences (C-NIP) in Chinese lay clients and mental health professionals: Factor structure, measurement invariance, and scale differences.

“…However, the fit statistic thresholds suggested by Hu and Bentler (1999) were based on simulation research and might have limited generalizability to the measurement model in applied research (Brown, 2015). Many clinical measures also adopt a more liberal CFI criterion (e.g., CFI > .90; Coleman et al, 2022; Klocek et al, 2022). Moreover, Brown (2015) noted that it might be less of a concern if other indices indicate a “good” model fit when one fit index is suboptimal.…”

What Makes Children's Responses to Creativity Assessments Difficult to Judge Reliably?

“…In other areas of empirical work within the psychological and education sciences, it is common to employ close-ended and selected-response type measures such as multiple-choice knowledge tests (e.g., McMullen, Hannula-Sormunen, Lehtinen, & Siegler, 2022) or Likert-style self-report measures (e.g., Coleman, Dong, Dumas, Owen, & Kopta, 2022). Although creativity researchers have fruitfully utilized such selected-response assessments (e.g., the Remote Associates Test; Mednick, 1962), creativity research in general is characterized by the collection and interpretation of open-ended and ill-structured data wherein tasks and assessments are administered to participants, and participants respond to those stimuli by positing ideas in either a spoken, written, or drawing format (Barbot, Hass, & Reiter-Palmon, 2019).…”