Assessing predictions of violence: Being accurate about accuracy.

Abstract

The prediction of violence occupies a prominent and controversial place in public mental health practice. Productive debate about the validity of violence predictions has been hampered by the use of methods for quantifying accuracy that do not control for base rates or biases in favor of certain outcomes. This article describes these problems and shows how receiver-operating characteristic analysis can be used to solve them. The article also reanalyzes 58 data sets from 44 published studies of violence prediction. Taken together, these data strongly suggest that mental health professionals' violence predictions are substantially more accurate than chance. Short-term (1-7 day) clinical predictions seem no more accurate than long-term (> 1 year) predictions. Past behavior alone appears to be a better long-term predictor of future behavior than clinical judgments and may also be a better indicator than cross-validated actuarial techniques.

The effects of rational and experiential information processing of expert testimony in death penalty cases

“…This previous research has shown clinical testimony to be more persuasive than actuarial testimony, despite the overall inferiority of clinical testimony in predicting violence (Borum, 1996;Gardner et al, 1996;Mossman, 1994;Quinsey et al, 1998). According to cognitive-experiential self-theory (CEST), this pattern of results may be due to the relative influence of experiential over rational information processing.…”

“…the type of the prediction offered by the expert psychiatrist in the Barefoot case) are unreliable (Grisso & Appelbaum, 1992;Faust & Ziskin, 1988;Hart, Webster, & Menzies, 1993;Melton et al, 1997;Monahan, 1981;Monahan & Steadman, 1994;Otto, 1992;Showalter, 1990), more recent research suggests they are not as inaccurate as was portrayed in the Barefoot case (Mossman, 1994; his meta-analysis suggests false positive rates closer to 44%), and clinical predictions may be equal to other methods of dangerousness prediction for very short time periods (McNeil, Sanders, & Binder, 1998;Mossman, 1994).…”

“…ignoring the base rate, assigning a non-optimal weight to this factor, and utilizing the representativeness heuristic) in arriving at a conclusion that the defendant will be dangerous in the future. As a result of these errors, for the long-term predictions of dangerousness required by the Texas death penalty statute as well as most jurisdictions' sexually violent predator laws, clinical predictions have been demonstrated to be uniformly inferior to other means of violence prediction (see, e.g., Borum, 1996;Gardner, Lidz, Mulvey, & Shaw, 1996;Mossman, 1994;Quinsey et al, 1998).…”

Assessing risk of reoffending in adolescents who have committed a sexual offense: the accuracy of clinical judgments after completion of risk assessment instruments

“…post-release sexual or non-sexual violence; Mossman, 1994). Thus, receiver operating characteristic (ROC) analyses were also performed to evaluate the ability of instrumentinformed clinical judgments to predict recidivism, as well as to evaluate the predictive validity of these judgments across confidence groups.…”

Using the Violence Risk Appraisal Guide (VRAG) to Predict In-Prison Aggressive Behavior in a Swiss Offender Population

“…Even though many studies have been able to demonstrate the superior predictive accuracy of both actuarial methods (Bonta, Law, & Hanson, 1998;Grove, Zald, Lebow, Snitz, & Nelson, 2000;Hilton & Simmons, 2001;Mossman, 1994;Rice & Harris, 1997;Swets, Dawas, & Monahan, 2000) and systematic content-relevant self-report procedures (Walters, 2006) over clinical and intuitive methods, most mental health professionals in Germany, Switzerland, and Austria still use these methods (Urbaniok et al, 2006).…”