Assessing the Mental Health of Maltreated Youth with Child Welfare Involvement Using Multi-Informant Reports

Makol, Bridget A.; Reyes, Andres De Los; Garrido, Edward F.; Harlaar, Nicole; Taussig, Heather N.

doi:10.1007/s10578-020-00985-8

Cited by 16 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, as in prior work (Achenbach et al, 1987;De Los Reyes et al, 2015), we observed low-to-moderate levels of correspondence, with informants observing behavior in different contexts (i.e., parents and UUOs) displaying particularly low levels of correspondence (Table 2). In line with recent work (e.g., De Los Reyes et al, 2016;Lerner et al, 2017;Makol et al, 2019Makol et al, , 2021, we also found that the patterns of reports within a given set of informants (e.g., parent > teacher; youth < parent) tended to operate similarly across rated domains. In fact, when we applied the Satellite Model to integrating reports taken from adolescents, parents, and UUOs, the eigenvalues and component loadings were remarkably similar across the domains measured with these reports (i.e., social anxiety, avoidance behaviors, fears of negative evaluation, depression, and impairments; see Table 3).…”

Section: Main Findingssupporting

confidence: 91%

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

et al. 2022

Self Cite

View full text Add to dashboard Cite

Accurately assessing youth mental health involves obtaining reports from multiple informants who typically display low levels of correspondence. This low correspondence may reflect situational specificity. That is, youth vary as to where they display mental health concerns and informants vary as to where and from what perspective they observe youth. Despite the frequent need to understand and interpret these informant discrepancies, no consensus guidelines exist for integrating informants’ reports. The path to building these guidelines starts with identifying factors that reliably predict the level and form of these informant discrepancies, and do so for theoretically and empirically relevant reasons. Yet, despite the knowledge of situational specificity, few approaches to integrating multi-informant data are well-equipped to account for these factors in measurement, and those that claim to be well-positioned to do so have undergone little empirical scrutiny. One promising approach was developed roughly 20 years ago by Kraemer and colleagues (2003). Their Satellite Model leverages principal components analysis (PCA) and strategic selection of informants to instantiate situational specificity in measurement, namely components reflecting variance attributable to the context in which informants observe behavior (e.g., home/non-home), the perspective from which they observe behavior (e.g., self/other), and behavior that manifests across contexts and perspectives (i.e., trait). The current study represents the first construct validation test of the Satellite Model. A mixed-clinical/community sample of 134 adolescents and their parents completed six parallel surveys of adolescent mental health. Adolescents also participated in a series of simulated social interactions with research personnel trained to act as same-age, unfamiliar peers. A third informant (unfamiliar untrained observer) viewed these interactions and completed the same surveys as parents and adolescents. We applied the Satellite Model to each set of surveys and observed high internal consistency estimates for each of the six-item trait (α = 0.90), context (α = 0.84), and perspective (α = 0.83) components. Scores reflecting the trait, context, and perspective components displayed distinct patterns of relations to a battery of criterion variables that varied in the context, perspective, and source of measurement. The Satellite Model instantiates situational specificity in measurement and facilitates unifying conceptual and measurement models of youth mental health.

show abstract

Section: Main Findingssupporting

confidence: 91%

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Future research should administer behavioral measures (such as the Need Threat Scale) following each run to better understand the implications of the identified neural patterns. Finally, it is important to note limitations with retrospective assessments of adversity, which may identify different mechanisms underlying disease risk compared to prospective assessments ( Baldwin et al, 2019 ), as well as caregiver-reports of mental health problems as concordance across youth, caregiver and teacher reports varies by problem type (internalizing vs externalizing) and contexts (e.g., home vs school; Makol et al 2021 ).…”

Section: Discussionmentioning

confidence: 99%

Differential neural sensitivity to social inclusion and exclusion in adolescents in foster care

Vijayakumar

Cheng

Flannery

et al. 2022

NeuroImage: Clinical

View full text Add to dashboard Cite

“…Recent meta-analyses show that parent and teacher reports of preschool children’s disruptive behavior display low-to-moderate levels of correspondence ( Carneiro et al, 2021 ). Prior studies also highlight that parent-teacher dyads display considerable heterogeneity in patterns of reporting across developmental periods, as some dyads disagree in their reports, whereas other dyads agree (see Fergusson et al, 2009 ; Lerner et al, 2017 ; Sulik et al, 2017 ; Makol et al, 2021 ). In line with the notion of situational specificity, De Los Reyes et al (2009) compared patterns of parent and teacher reports of disruptive behavior against independent assessments of children’s actual behavior (i.e., using the Disruptive Behavior Diagnostic Observation Schedule; Wakschlag et al, 2010 ).…”

Section: Measurement Invariance: the Wrong “Sentence” For Multi-infor...mentioning

confidence: 99%

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments

et al. 2022

Self Cite

View full text Add to dashboard Cite

On page 1 of his classic text, Millsap (2011) states, “Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are irrelevant [emphasis added] to the attribute being measured.” By construction, measurement invariance techniques require not only detecting varied conditions but also ruling out that these conditions inform our understanding of measured domains (i.e., conditions that do not contain domain-relevant information). In fact, measurement invariance techniques possess great utility when theory and research inform their application to specific, varied conditions (e.g., cultural, ethnic, or racial background of test respondents) that, if not detected, introduce measurement biases, and, thus, depress measurement validity (e.g., academic achievement and intelligence). Yet, we see emerging bodies of work where scholars have “put the cart before the horse” when it comes to measurement invariance, and they apply these techniques to varied conditions that, in fact, may reflect domain-relevant information. These bodies of work highlight a larger problem in measurement that likely cuts across many areas of scholarship. In one such area, youth mental health, researchers commonly encounter a set of conditions that nullify the use of measurement invariance, namely discrepancies between survey reports completed by multiple informants, such as parents, teachers, and youth themselves (i.e., informant discrepancies). In this paper, we provide an overview of conceptual, methodological, and measurement factors that should prevent researchers from applying measurement invariance techniques to detect informant discrepancies. Along the way, we cite evidence from the last 15 years indicating that informant discrepancies reflect domain-relevant information. We also apply this evidence to recent uses of measurement invariance techniques in youth mental health. Based on prior evidence, we highlight the implications of applying these techniques to multi-informant data, when the informant discrepancies observed within these data might reflect domain-relevant information. We close by calling for a moratorium on applying measurement invariance techniques to detect informant discrepancies in youth mental health assessments. In doing so, we describe how the state of the science would need to fundamentally “flip” to justify applying these techniques to detect informant discrepancies in this area of work.

show abstract

Assessing the Mental Health of Maltreated Youth with Child Welfare Involvement Using Multi-Informant Reports

Cited by 16 publications

References 35 publications

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

Differential neural sensitivity to social inclusion and exclusion in adolescents in foster care

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments

Contact Info

Product

Resources

About