Comparative Validity of Two Preschool Screening Tests

Abstract

Screening tests have the explicit purpose of assigning individuals an initial classification (ie, at risk or not at risk) in a sequential assessment process. Although screening test validity studies generally present correlational validity coefficients based on continuous scores, validity figures based on classificational scores', which summarize the proportion of accurate and inaccurate predictions, are more relevant to the decision-making function of screening tests. Correlational and classificational approaches were compared in a concurrent validity study of 40 preschool children who were given two screening tests (Denver Developmental Screening Test and Developmental Indicators for the Assessment of Learning) and three criterion measures (Stanford-Binet Intelligence Scale, Peabody Picture Vocabulary Test, and Woodcock-Johnson Psycho-Educational Battery). Correlations between the two screening tests and between the screening and criterion measures were inconsistent with prediction rates based on classificational screening test scores. On the basis of correlational validity coefficients, relationships between measures in this study would have been overestimated. E ducational and psychological tests, either alone or in combination with other devices and procedures, are used to gather data for psychoeducational decision making. Clearly it is important to evaluate the extent to which such test applications contribute to accurate and effective decision making. Yet, it is rare to find test validity presented in terms of classificational hit rate, i.e., with a contingency table indicating the frequency of accurate predictions (true positives, true negatives) and inaccurate predictions (false positives, false negatives).Failure to express validity data in terms of classificational hit rate is of particular concern in the case of screening tests (Salvia & Ysseldyke 1977, Lichtenstein 1979). Screening measures provide an excellent example of tests which are explicitly intended for decision making. The aim of a screening test is to select those persons who are at risk for a particular condition, and therefore, warrant more comprehensive evaluation. To determine whether the value of a screening test is sufficient to justify its use in a screening program, the prospective user needs validity data indicating the predictive efficiency of classificational test scores, i.e., whether the test selects persons as intended. The point is addressed both generally and specifically in Stan- dards for Educational and Psychological Tests (American Psychological Association, American Educational Research Association, and American Council on Measurement in Education 1974):Statements about validity should refer to the validity of particular interpretations or of particular types of decisions (p 31). Errors of prediction should be estimated and reported. . . . For a dichotomous criterion, this objective might be achieved by indicating the proportion of hits, misses, and false inclusions at various cutting scores (p 40).Despite the imperati...

Evaluating Potential Test Components for a New Cognitive Screening Test: a Preliminary Study

“…It must also be brief (Satz & Fletcher, 1982). A similar position has been expressed by others (e.g., Lichtenstein, 1981;Lichtenstein & Ireton, 1984;Satz & Fletcher, 1988). The first series of grade school studies resulted in eliminating some tasks from the item pool.…”