Examining the predictive validity of behavior screeners across measures and respondents

Abstract

Through this study we compared different informants and different behavior screening tools that are available to screen for behavioral and emotional risk. We examined screening results from 100 students from a high school with a high achiever's magnet program in the Southeastern United States (school demographics: 71% female and 79% black, non-Hispanic). This school conducted behavior screening using both teacher-report via the Student Risk Screening Scale, Internalizing/Externalizing (SRSS-IE) and student report via the Strengths and Difficulties Questionnaire (SDQ). Nonparametric correlation tests were conducted between the SRSS-IE and SDQ externalizing and internalizing scores. Generalized linear regression models were created based on the data (demographics, internalizing, and externalizing scores on both screening tools) to model the two count outcomes (office discipline referrals [ODRs] and absences). Diagnostic accuracy metrics were generated from the scores on both the SRSS-IE and SDQ with the outcome measures (ODRs and absences). Externalizing scores on both screening tools had statistically significant low correlations. The ODR model contained two predictors: externalizing score on the SRSS-IE and internalizing score on the SDQ, while the absence model contained grade and the externalizing score on the SDQ.The highest accuracy and agreement values were seen between students with elevated risks on both screening measures and ODRs. The results confirmed that teacher and student reporting, as well as different screening tools, will result in some different students being identified. The choice of informant and screening tools should be dependent on the needs and resources of the school. K E Y W O R D S internalizing and externalizing, school-based behavior screening, Strengths and Difficulties Questionnaire, Student Risk Screening Scale

Differential Item and Test Functioning of the SRSS-IE12 Across Race, Ethnicity, Gender, and Elementary Level

“…Another important area of future research is to determine whether the SRSS-IE is better than ODRs at identifying children with externalizing concerns. In a separate study, our research team found the fall SRSS-IE12 results predicted year-end ODRs, suggesting that students can be identified and receive intervention prior to accumulating numerous disciplinary referrals (Jones et al, 2020). This study did not examine this relationship, but future research might extend it to consider the intervention validity of screening tools like the SRSS-IE12.…”

Prediction of child and adolescent outcomes with broadband and narrowband dimensions of internalizing and externalizing behavior using the child and adolescent version of the Strengths and Difficulties Questionnaire

“…Lenzen et al [23] highlighted the association of greater conduct problems and emotional symptoms (narrowband scales) with increased school absenteeism. School absence was also linked to internalizing behavior (broadband scale) while school discipline referrals were related to externalizing behavior (broadband scale) [24]. Moreover, students with special educational needs, especially those with learning disabilities, have greater peer problems (narrowband scale) [12].…”

“…Although the originally proposed 5-factor structure of the SDQ (5 narrowband subscales) is often applied in research [ 12 , 17 , 19 , 20 , 22 , 23 ], the narrowband subscales are sometimes subsumed into broadband behavior scales (internalizing and externalizing behavior) [ 4 , 14 , 15 , 18 , 21 , 24 ], which has resulted in an ongoing controversy, not only in educational research, regarding the factor structure of the SDQ [ 25 – 27 ]. This controversy implies a discussion about the usefulness of broadband and narrowband scales of behavior.…”

Examination of the Factor Structure and Measurement Invariance of the SRSS-IE

“…Results indicated sixth- to twelfth-grade students with high levels of risk per fall SRSS-IE scores were likely to have lower grade point average (GPA), fail more courses, have more nurse visits, and spend more time out of class for in-school suspensions compared to students with low-risk scores for externalizing and internalizing behaviors. Similarly, Jones, Graybill, Barger, and Roach (2020) also reported SRSS-E7 scores to be significantly predictive of end-of-year ODR counts (B = .23 [ p < .0001]) and total absences (B = .13 [ p < .001]) for high school students.…”