Applicability and Efficiency of a Computerized Adaptive Test for the Washington Assessment of the Risks and Needs of Students

Abstract

The Washington Assessment of the Risks and Needs of Students (WARNS) is a computer-based assessment created to help courts, schools, and youth service providers determine an adolescent’s risks and needs that may lead to truancy, drop out, or delinquency from school. Users are advised to consider the WARNS total score to work with youth. A total score estimate based on fewer items than the full item set may result in less respondent burden, administration time, and fatigue, while not hindering accurate decisions. This simulation study examined the applicability and efficiency of a computerized adaptive test (CAT) to estimate a WARNS total score under a unidimensional item response theory model. The results demonstrate that the CAT provides an accurate estimate of students’ risks and needs and reduces the number of items administered for each examinee compared with the existing version. Future directions and limitations of CAT development with the WARNS are discussed.

Simulated computer adaptive testing method choices for ability estimation with empirical evidence

“…Also worthy of note is the fact that Bayesian estimator are convenient with small-scaled tests and when ability levels are not extremely low [17]. This shows that CAT can be applied to school-based assessments rather than only standardized [31].…”

Applicability and Efficiency of a Polytomous IRT-Based Computerized Adaptive Test for Measuring Psychological Traits

“…The SCI factors whose adjacent eigenvalue ratios are greater than 3 indicate unidimensionality. When the assumption of unidimensionality is met, the assumption of local independence is also met because only one factor affects the person's responses to the items (Crocker & Algina, 1986;Hambleton et al, 1991;Embretson & Reise, 2000;Thissen & Wainer, 2001;Reise & Revicki, 2015). For model selection, the -2LL values for the GRM, GRM-C, GPCM, and GPCM -C models were determined using the ltm (1.0) package (Table 1).…”

“…Responses are considered valid as long as individuals answer sincerely. However, because the test duration is quite long for some measurement instruments, the person's motivation to respond may decrease, and the validity of the measurements may be negatively affected (Crocker & Algina, 1986;Gardner et al, 2004). This situation, seemingly related only to the usefulness of the measurement instrument, also raises validity issues.…”