Optimizing Youth Risk Assessment Performance: Development of the Modified Positive Achievement Change Tool in Washington State

Abstract

The Positive Achievement Change Tool (PACT) was developed in 1997 using a theoretical construction of items, responses, and weights. While derived from an original tool created for a Washington State probation population, the risk-need assessment is one of the most widely used youth tools utilized today. To advance the model from its theoretical construction, the current study demonstrates tool updates making use of a large sample of Washington State youth ( N = 50,862). Specifically, several mechanisms were utilized to customize the assessment, including (a) item weighting, (b) outcome specificity, and (c) gender responsivity. Based on the updated design, we identify improvements in predictive validity of the continuous risk scale, accuracy of risk-level assignment, and reductions in racial/ethnic disparity. Scheduled to be implemented in the coming year, this article describes the development of the Modified Positive Achievement Change Tool (M-PACT).

Redesigning the Central Eight: Introducing the M-PACT Six

“…The current study extends previous validation findings (Hamilton et al, 2019, 2020), providing empirical support for the MPACT’s needs assessment development. We aimed to construct a youth-specific, stand-alone needs assessment that can be paired with the optimized risk tool.…”

“…We note that, while valuable to assess, content validity is a subjective measure, requiring direct input from subject matter experts’ (SME) and not feasibly tested under the current study conditions. Second, predictive and concurrent validity of the MPACT have been previously assessed (Hamilton et al, 2019, 2020), and results are not restated here. In the present study, we focus analyses on convergent/divergent and internal (latent structure) validity.…”

“…Yet, despite intentions to update the tool after sufficient data had been collected (Barnoski, 2004), the PACT’s theoretical needs domains have gone relatively unchanged since its 1997 conception. However, recent updates to the risk tool resulted in a modified version, or MPACT, which has demonstrated improved predictive strength (Hamilton et al, 2019; Hamilton et al, 2020).…”

“…The current study utilizes a sample collected as part of a larger study completed in Washington State (Hamilton et al, 2019). The sample included youth that completed a full PACT assessment provided by the Washington State Administrative Office of the Courts.…”

“…As we described, the field has focused on evaluating predictive validity. While predictive validity is important in determining the effectiveness of a risk assessment tool and has been assessed previously for the MPACT (Hamilton et al, 2019, 2020), a needs assessment is constructed to meet differing demands of the RNR model. Domains must be developed to assess their latent connection.…”