Psychometrics and Measurement Invariance of the Emotional and Behavioral Screener

Abstract

Many students who exhibit behavioral and emotional problems during adolescence often show less severe problems in school in early grades. Screening for these early indicators can help educational professionals direct support to students who are more likely to benefit from increased support. The screening protocol needs to be psychometrically sound, accurate in identifying students at risk, and resource efficient. The present study explored the psychometric properties of a brief behavioral screener-the 10-item Emotional and Behavioral Screener (EBS)-for a sample of 5,109 first-grade students assessed as part of a universal screening process. Confirmatory factor analysis and Rasch measurement techniques were used in conjunction to evaluate the factorial validity, item quality, rating scale functioning, and measurement invariance across gender of the EBS. Results indicated that the EBS has largely strong psychometric properties, with a few notable exceptions: potential multidimensionality, significant differential item functioning across gender, and only moderate differentiation between rating-scale categories (i.e., response options). Limitations and directions for research to clarify some remaining questions about this screening procedure are also discussed.

Latent Structure of Scores From the Emotional and Behavioral Screener

“…Although one previous study reported evidence of the factor structure and reliability of the EBS scores (e.g., Lambert, Epstein, Ingram, et al, 2014), the factor model goodness-of-fit was poor and the study only included students in first grade. The importance of establishing a tenable measurement model for the EBS scores cannot be overstated because the measurement model serves as the "basis and rationale for arriving at the composite [scores]" (American Educational Research Association, American Psychological Association, & National Council of Measurement in Education, 1999, p. 20), and is a prerequisite for assessing score reliability (Slaney & Maraun, 2008).…”

“…In an effort to support the construct validity of EBS scores, Lambert, Epstein, Ingram, Simpson, and Bernstein (2014) investigated the factor structure of scores from the EBS with a sample of first-grade students. They found some support for the internal structure of the EBS scores, but the confirmatory factor analysis (CFA) model demonstrated relatively poor fit to single-factor solution.…”

Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener

“…Since coefficient alpha was estimated separately for males and females, those independent estimates were compared to one another using the equations given by Feldt (1969). In addition to computing coefficient alpha, we also calculated coefficient omega (x; McDonald 1978) which is a confirmatory factor analysisbased estimate of reliability that is less biased when the internal structure of an assessment is not tau equivalent (i.e., the factor loadings differ across items) (Brown 2006;Green and Hershberger 2000) as the results of Lambert et al (2014b) suggested.…”

“…While initial research has been positive, researchers have stated the need for continued investigation of the technical characteristics of the EBS to ensure its appropriateness for use with a wide range of school-aged students (Lambert et al 2014b), as well as the acceptability of school personnel for using EBS as part of a screening process (Nordness et al 2014). In this study, we assessed the convergent and criterion validity of the EBS scores and the acceptability of the EBS as a behavioral screening tool within school-wide systems.…”

The impact of English language learner status on screening for emotional and behavioral disorders: A differential item functioning (DIF) study

“…A student is considered to be at risk if their total EBS score meets or exceeds the cutoff score. Although the EBS is a relatively new instrument, EBS scores have demonstrated adequate reliability and validity during the test development process (Cullinan & Epstein, ) and in subsequent research (e.g., Lambert, Epstein, Ingram, Simpson, & Bernstein, ; Lambert, January, & Pierce, ). However, researchers have yet to examine whether EBS scores are unbiased when assessing the emotional and behavioral risk of ELL students.…”