The case against categorical risk estimates

Abstract

Risk estimates can be communicated in a variety of forms, including numeric and categorical formats. An example of the latter is “low/medium/high risk.” The categorical format is preferred by judges and practitioners alike, and is mandated by the most commonly utilized forensic risk assessment instruments (the HCR‐20 and the Static‐99). This article argues against the practice of communicating risk in categorical terms on empirical and normative grounds. Empirically, there is no consensus about what level of risk corresponds to a particular category, such as “high risk.” Moreover, recent studies indicate that categorizing an otherwise continuous risk estimate does not add incremental predictive validity to the risk estimate. Normatively, categorization obscures what is fundamentally a value judgment about the relative costs and benefits of correct (e.g., true positive) and incorrect (e.g., false positive) outcomes. Such a judgment is inherently non‐scientific and invades the province of the jury. Indeed, categorical risk estimates are in principle no different than “dangerousness predictions,” which are simply binary and which have been denounced by the field. The fact that alternative risk communication formats have limitations does not justify continuing the pervasive practice of communicating categorical risk estimates.

Risk assessment communication difficulties: An empirical examination of the effects of categorical versus probabilistic risk communication in sexually violent predator decisions

“…Categorical risk assessments usually consist of statements that a person is “high,” “medium,” or “low” risk, or something similar. This type of risk communication has been criticized for its: (i) lack of consistent definition and the number of categories across different risk assessment instruments; (ii) lack of consistency when experts interpret and communicate risk in legal proceedings; (iii) likelihood of leading decision‐makers to overestimate rates of violence (e.g., by not providing base rate information for recidivism, which is lower than most decision‐makers believe); and (iv) likelihood of being interpreted to have definitively answered legal questions (i.e., a “high” risk offender must be “likely” to recidivate) (see Scurich, for further discussion of all these issues).…”

“…At the heart of these dispositions is whether the respondent is “likely” to engage in future acts of sexual violence. Expert testimony is the common means of presenting this risk information to the jury; in fact, several jurisdictions require the use of risk assessment instruments by mental health professional to determine an individual's risk of sexual recidivism (e.g., California) (Scurich, ).…”

“…Currently within the psycholegal field there is considerable debate concerning the most effective means to convey risk prediction expert testimony to the trier of fact (Scurich, ). Some researchers believe that risk information should be conveyed in categorical terms (e.g., “X poses a {high, moderate, or low or some other categorical designation} risk”) (e.g., Evans & Salekin, ).…”

“…Others argue that numerical‐based predictive probabilities based on a chosen risk assessment instrument's normative sample should be routinely employed (e.g., “X belongs to group in which Y % of individuals' recidivated over a period of Z years”) (e.g., Hilton, Carter, Harris, & Sharpe, ). This controversy has been exacerbated by the development and widespread use of risk assessment based on structured professional judgment (SPJ) instruments like the HCR‐20 V3 (Douglas et al, ), which do not allow for the presentation of numerical risk information because they were not developed with normative samples, and thus require categorical risk communication (Scurich, ).…”

“…Furthermore, judges appear to prefer to receive risk information in this manner, finding categorical presentation more probative than numerically based estimates (Evans & Salekin, ; Evans & Salekin, ; Kwartner, Lyons, & Boccaccini, ). Yet, to date there simply is not sufficient empirical research to suggest consensual best practices or professional standards for risk communication (Scurich, ). The present study is intended to inform this controversy as well as best practices by examining how mock jurors evaluate different forms of risk communication (categorical vs. probabilistic) in a simulated sexual violent predator (SVP) civil commitment hearing.…”