Considerations for evaluating universal screening assessments

Screening instruments in child and adolescent psychiatry: general and methodological considerations

“…(2) Internal consistency demands that items of the screener or its subscales are homogenous, i.e., measure the same construct(s). (3) If the screen is an interview, inter-rater reliability evaluates the rate of agreement between different raters [12,13].…”

“…Three complimentary main aspects of validity are commonly required for screening instruments: (1) of main interest for a clinical-diagnostic application is the criterion validity; it is an indicator of how well a screener's result corresponds to an individual result on a specified criterion [12,13]. Thereby, two aspects of criterion validity are distinguished in relation to the time between the assessment of screener and criterion: (a) the degree to which the screener can identify individuals who currently have any or a specific psychiatric disorder (concurrent validity; requires nearly simultaneous assessment of screener and criterion in the test construction phase, while, later in practice, some time might pass between screening and formal psychiatric assessment), and (b) the extent to which an individual's score on a screener will accurately predict the individual's future result such as a psychiatric disorder (predictive validity; outcome criterion will reveal only in future and is assessed considerably later than screener) [12,13]. (2) When the focus is less on the result of a screen but rather on its score and the measure of interest is less well defined than, for example, a formal diagnosis but relates to a construct that is not directly assessable (such as intelligence or personality characteristics), the construct validity is assessed.…”

“…One aspect of the construct validity is the convergent validity, which is good when the correlation between the screener and another established assessment of the same construct is high. The opposite aspect of construct validity is the discriminant validity, which is high when screener scores do not correlate with measures of other constructs [12]. For example, scores of an ADHD screener should not be highly positively correlated to scores of scales assessing oppositional defiant/conduct or emotional disorders.…”

Protective factor screening for prevention practice: Sensitivity and specificity of the DESSA-Mini.

“…The field of school psychology has contributed practice standards to guide universal screening in schools and child-care/after-school settings, particularly regarding their appropriateness for intended use, technical adequacy, and their usability in schools and after-school contexts (Glover & Albers, 2007;Shapiro, Accomazzo, Claassen, & Fleming, 2015). General guidelines are also articulated in the Standards for Educational and Psychological Testing (Joint Committee on Standards for Educational and Psychological Testing, 2014).…”

“…Common metrics for determining criterion validity include determinations of sensitivity, specificity, positive predictive value, and negative predictive value (Glover & Albers, 2007). Sensitivity is a determination of the extent to which a screening instrument correctly identifies those who are actually at risk.…”