Diagnostic Utility of the Social Skills Improvement System Performance Screening Guide

Abstract

Researchers investigated the diagnostic utility of the Social Skills Improvement System: Performance Screening Guide (SSIS-PSG). Correlational, regression, receiver operating characteristic (ROC), and conditional probability analyses were run to compare ratings on the SSIS-PSG subscales of Prosocial Behavior, Reading Skills, and Math Skills, to report card grades for conduct, reading, and math, respectively. Respective subscales were all statistically significantly correlated with one another. In addition, all regressions indicated significant predictions for the SSIS-PSG to respective report card grades. ROC analyses for SSIS-PSG Math with math grades and SSIS-PSG Reading with reading grades were statistically significant and described as fair. ROC analysis for SSIS-PSG Prosocial Behavior with conduct grades was not significant and described as poor. In a conditional probability analysis, the variable of concern for screeners concerns false negative ratios; all estimates for this fell within the targeted range.

SSIS Performance Screening Guide as an Indicator of Behavior and Academics: A Meta-Analysis

“…However, divergent validity was less substantiated based on these analyses (i.e., some of the items were also significantly correlated with less similar variables, such as Motivation to Learn being most strongly correlated with broadband internalizing/externalizing problems). Of course, prospective studies (e.g., Kettler et al, 2012; Kilgus et al, 2012; Krach et al, 2017) are needed to evaluate the ability of PSG items to facilitate accurate decision-making and to identify students in need of further services. For now, the results of this meta-analysis suggest that based on current research, scores on the four PSG items are significantly related to important academic and behavioral variables among students in elementary school settings, and therefore hold potential for contributing to universal screening approaches.…”

“…Inter-rater reliability, based on a comparison of an elementary student’s teacher and another person familiar with the student, yielded results in the moderate score reliability range (.55–.68). However, less information is available in the test manual regarding the validity of PSG scores; this lack of documented support has been identified as a significant weakness (Jenkins et al, 2014; Krach et al, 2017). The manual does not describe how items were developed and does not provide evidence of content validity specific to the PSG.…”

The Utility of Receiver Operating Characteristic Curve in Educational Assessment: Performance Prediction

“…However, few research have been conducted applying ROC despite the advantage of giving quality information regarding the classification. It is not as widely used to evaluate students' knowledge and skills [13] though the ROC has been widely used in various disciplines including medical or clinical settings for diagnostic purposes [14][15][16][17]. As such, the goal of this paper is to introduce the ROC graph as a means to overcome the limitations of CA and CC indices in educational assessment.…”