Multilevel Confirmatory Factor Analysis of a Scale Measuring Interagency Collaboration of Children’s Mental Health Agencies

Abstract

Multilevel confirmatory factor analysis was used to evaluate the factor structure underlying the 12-item, three-factor Interagency Collaboration Activities Scale (IACS) at the informant level and at the agency level. Results from 378 professionals (104 administrators, 201 service providers, and 73 case managers) from 32 children's mental health service agencies supported a correlated threefactor model at each level and indicated that the item loadings were not significantly (p < .05) different across levels. Reliability estimates of the three factors (Financial and Physical Resource Activities, Program Development and Evaluation Activities, and Collaborative Policy Activities) at the agency level were .81, .60, and .72, respectively, while these estimates were .79, .82, and .85 at the individual level. These multilevel results provide support for the construct validity of the scores from the IACS. When the IACS was examined in relation to level-1 and level-2 covariates, results showed that participants' characteristics (i.e., age, job role, gender, educational level, and number of months employed) and agency characteristics (i.e., state location and number of employees) were not significantly (p > .05) related to levels of interagency collaboration.Interagency collaboration has been defined broadly as "mutually beneficial and well-defined relationships entered into by two or more organizations to achieve common goals" (Mattessich, Murray-Close, & Monsey, 2001, p. 4). Additional defining characteristics of interagency collaboration have included: (a) developing and agreeing to a set of common goals and directions, (b) sharing responsibility for obtaining those goals, and (c) working together at all levels of an organization to achieve those goals (Bruner, 1991;Cumblad, Epstein, Keeney, Marty, & Soderlund, 1996).In the past two decades, the call for collaboration among child-serving organizations has increased as many believe that important problems faced by children that result from being served by multiple agencies (e.g., service fragmentation, gaps, barriers) cannot be resolved effectively by single entities working alone (Bergstrom et al., 1995; Mattessich et al., 2001;Salmon, 2004). For example, recent reforms in children's mental health service delivery, such as the systems of care approach, have emphasized interagency collaboration as an important element in providing comprehensive services to children with serious emotional disturbance (Stroul & Friedman, 1986). Interagency collaboration may provide a way to cope with increasing complexity; meet expanding expectations, needs, and demands of human services; maximize human resources; share facilities and program resources; and improve utilization of funds and personnel (Jones, Thomas, & Rudd, 2004;Lippitt & Van Til, 1981).While many agree about the value of interagency collaboration, others have identified potential negatives associated with interagency collaboration. These negatives include diffusion of responsibility, reduced service quality, and e...

Authoritative School Climate and Student Academic Engagement, Grades, and Aspirations in Middle and High Schools

“…These studies used methodological advances in multilevel modeling that are relatively new in the development of measurement scales (Dedrick & Greenbaum, 2011). Employing both exploratory and confirmatory factor analyses, the studies identified sets of items that measured disciplinary structure, student support, and student engagement at both student and school levels of analysis.…”

Using Multilevel Factor Analysis With Clustered Data

“…Generally, finding a two-correlated factor model would provide evidence to show that a unidimensional construct of positive values was not present and that an overall summed score should not be interpreted. However, current findings illustrate how factor models may differ depending on the level of analysis, in line with concept that measured constructs may have different meanings at the different levels of analysis (Bliese, 2000;Dedrick & Greenbaum, 2011;D'Haenens et al, 2010;Schweig, 2013). For applied researchers, it is important to keep in mind that some constructs may be meaningful at the individual level (e.g., personality assessments), at the group level (e.g., diversity), or both (Reise et al, 2005).…”

“…respondents per school and the ICC is the intraclass correlation of the factor that reflects the amount of variability resulting from the school level (Dedrick & Greenbaum, 2011). Given the average cluster size of 46.53 and an overall ICC of .027, the reliability of the PVS at Level 2 was .92, reflecting high reliability.…”

Development and Initial Validation of a Scale Measuring the Beliefs of Educators Regarding Response to Intervention

“…Although the literature on teacher beliefs and on RTI models contributed to item development, the lack of empirical information on beliefs associated with RTI indicated a need to explore the factor structure of the instrument. In addition, research indicates that the factor structure of an instrument can vary across units of analysis (Dedrick & Greenbaum, 2011), an important assertion considering that the Beliefs on RTI Survey was designed to examine beliefs at the school level by collecting data on the beliefs of individual educators. Psychometric studies have traditionally focused on investigating the factor structure of an instrument at the person level only (Dedrick & Greenbaum, 2011), thereby ignoring the potential effects associated with individuals being nested within organizations (the influence of school level variables on individual educators).…”

“…In addition, research indicates that the factor structure of an instrument can vary across units of analysis (Dedrick & Greenbaum, 2011), an important assertion considering that the Beliefs on RTI Survey was designed to examine beliefs at the school level by collecting data on the beliefs of individual educators. Psychometric studies have traditionally focused on investigating the factor structure of an instrument at the person level only (Dedrick & Greenbaum, 2011), thereby ignoring the potential effects associated with individuals being nested within organizations (the influence of school level variables on individual educators). Therefore, the examination of the internal structure of a measure of educators' beliefs must investigate the factor structure and reliability at the school level, not just the educator level.…”