Valid or voodoo? A qualitative study of attorney perceptions of risk assessment in sentencing and plea bargaining

Abstract

Prior research largely has explored judicial perceptions of risk assessment in sentencing. Little is known about how other court actors, specifically, prosecutors and defense attorneys, perceive risk assessments in the sentencing process. Here, we report a qualitative study on the use of risk assessment by prosecutors and defense attorneys in Virginia. A prior survey (n = 70) pointed to a statistically significant difference in how prosecutors and defense attorneys view the role of recidivism risk in sentencing. On the basis of the results of this quantitative study, we collected follow‐up qualitative data via interview (n = 30) to explain this unexpected difference. Analysis confirmed the survey findings that prosecutors and defense attorneys differ in their perceptions of risk assessment in sentencing. Results suggest that court actor perceptions vary as a function of professional role in the service of the identified client (the community or the defendant) and their interests. Although perceptions diverged on utility risk assessment in sentencing, both prosecutors and defense attorneys were outspoken in their skepticism of the Nonviolent Risk Assessment instrument that is used to predict recidivism risk in Virginia. This latter finding identifies obstacles that may emerge as jurisdictions adopt a risk‐based approach to sentencing. We conclude with recommendations for addressing these barriers that may provide useful guidance on the implementation process.

Risk Assessment in Sentencing

“…This finding was consistent with previous studies on the judicial perception of risk assessment at both sentencing (Metz et al, 2019;Monahan et al, 2018) and pretrial stages (DeMichele et al, 2018). Participants who were skeptical of an actuarial instrument were less likely to trust the results of a risk assessment, meaning that relevant risk information may be ignored in practice (Metz et al, 2019(Metz et al, , 2020.…”

“…On the whole, risk assessment instruments are more accurate than clinical judgment when it comes to predicting recidivism risk (Andrews et al, 2006;Grove et al, 2000;Latessa & Lovins, 2010). But research on perceptions of risk assessment in sentencing suggests that the main consumers of these instruments-judges, prosecutors, and defense attorneys-are skeptical of their predictive validity (Metz et al, 2019(Metz et al, , 2020Monahan et al, 2018). A recent study of prosecutors and defense attorneys by Metz et al (2020) found that while there was support for the use of risk assessment as a strategy for reducing mass incarceration, participants were practically unanimous in their objection to several risk factors used by the NVRA instrument-particularly gender and age.…”

“…But research on perceptions of risk assessment in sentencing suggests that the main consumers of these instruments-judges, prosecutors, and defense attorneys-are skeptical of their predictive validity (Metz et al, 2019(Metz et al, , 2020Monahan et al, 2018). A recent study of prosecutors and defense attorneys by Metz et al (2020) found that while there was support for the use of risk assessment as a strategy for reducing mass incarceration, participants were practically unanimous in their objection to several risk factors used by the NVRA instrument-particularly gender and age. This finding was consistent with previous studies on the judicial perception of risk assessment at both sentencing (Metz et al, 2019;Monahan et al, 2018) and pretrial stages (DeMichele et al, 2018).…”

“…If the main consumers of risk assessment information feel that the instruments are biased, invalid, or even "voodoo" (Metz et al, 2020), it is unlikely that risk assessment will achieve its purported ends of reducing incarceration without compromising public safety. The explosion of commercially available instruments with proprietary algorithms appears to only exacerbate the skepticism.…”

“…In doing so, Eaglin endorses machine learning (as opposed to regression models), expert and community input on recidivism and risk factors (as opposed to the black-box model of commercial instruments), and public information about the tool construction process (again in contrast to commercial instruments). These suggestions point to a disparity in what counts as "evidence" for recidivism, with instrument developers and policymakers seeing evidence as statistical probability while court actors' approach to evidence is perhaps more holistic: considering the contextual aspects of a case, a defendant's motivation, and the available community resources (Metz et al, 2020).…”