Maximizing measurement efficiency of behavior rating scales using Item Response Theory: An example with the Social Skills Improvement System — Teacher Rating Scale

Examining the Psychometric Properties of Maximally Efficient Items From the Social Skills Improvement System–Teacher Rating Scale

“…The SMIs identified by Anthony et al (2016) include 41 (58%) of the items on the Social Skills and Problem Behaviors scales from the full-length SSIS-TRS. Anthony et al (2016) found the SMIs to exhibit adequate internal consistency with Cronbach’s alpha ranging from .84 to .97 (median = .90) and stability coefficients ranging from .57 to .72 (median = .66; average interval length = 4.5 months) across scales and subscales. Furthermore, SMIs related as expected with other scores from academic and behavioral measures.…”

“…Furthermore, SMIs related as expected with other scores from academic and behavioral measures. Specifically, Anthony et al (2016) reported validity coefficients for Social Skills scale and subscale SMIs ranged from .11 to .36 (median = .27) for STAR Reading and Mathematics scores, and from .46 to .74 (median = .63) for ACES Motivation and Engagement scores. Furthermore, validity coefficients for Problem Behaviors scale and subscale SMI scores ranged from −.31 to −.14 (median = −.24) for STAR Reading and Mathematics scores, and from −.60 to −.19 (median = −.43) for ACES Motivation and Engagement scores.…”

“…To address this need, Anthony, DiPerna, and Lei (2016) utilized advanced methodology to identify a set of maximally efficient items (SMI) for each SSIS-TRS subscale. Specifically, they used item response theory (IRT; Hambleton, Swaminathan, & Rogers, 1991), an approach to scale development and refinement with particular advantages for improving measurement efficiency (e.g., Smith et al, 2000).…”

“…Consistent with this approach, Anthony et al (2016) evaluated item information functions (IIFs; visual representations of the measurement precision across levels of the target trait) to select SSIS-TRS items that increased measurement precision at lower levels of the measured trait to target students at risk for difficulties in SSIS-TRS domains. They also balanced item selection with considerations of domain coverage, as well as other statistical criteria (e.g., ensuring that IRT assumptions were met for the resulting SMIs).…”

“…Given the need for efficient measurement of social competence (Dirks, Treat, & Weersing, 2007; Whitcomb & Merrell, 2013), the purpose of this study was to examine the validity of the scores from the SSIS-TRS SMIs identified by Anthony et al (2016). Consistent with this goal, several convergent and discriminant hypotheses were tested in the current study.…”