Field reliability of competence to stand trial opinions: How often do evaluators agree, and what do judges decide when evaluators disagree?

Abstract

Despite many studies that examine the reliability of competence to stand trial (CST) evaluations, few shed light on "field reliability," or agreement among forensic evaluators in routine practice. We reviewed 216 cases from Hawaii, which requires three separate evaluations from independent clinicians for each felony defendant referred for CST evaluation. Results revealed moderate agreement. In 71% of initial CST evaluations, all evaluators agreed about a defendant's competence or incompetence (kappa = .65). Agreement was somewhat lower (61%, kappa = .57) in re-evaluations of defendants who were originally found incompetent and sent for restoration services. We also examined the decisions judges made about a defendant's CST. When evaluators disagreed, judges tended to make decisions consistent with the majority opinion. But when judges disagreed with the majority opinion, they more often did so to find a defendant incompetent than competent, suggesting a generally conservative approach. Overall, results reveal moderate agreement among independent evaluators in routine practice. But we discuss the potential for standardized training and methodology to further improve the field reliability of CST evaluations.

Competency to stand trial evaluations: A state‐wide review of court‐ordered reports

“…Similarly, clinicians found 19% of Alabama defendants IST (Cooper & Zapf, 2003), and 18% of federal defendants IST (Cochrane, Grisso, & Frederick, 2001). Clinicians found slightly higher rates of IST among Hawaii defendants (25% in Gowensmith et al, 2012) and much higher rates among a small sample of Utah defendants (53% in Skeem, Golding, Cohn, & Berge, 1998). However, most state-wide samples, at least historically, have suggested IST rates around the 18-25% range.…”

Tipping the scales of justice: the role of forensic evaluations in the criminalization of mental illness

“…16,18,22 Clearly, these threats to the reliability of forensic opinions must be addressed if evaluations are to reach the highest standards of objectivity and neutrality. The reliability of evaluator opinions is surprisingly low across nearly all psycholegal referral questions, 51 most likely for many of the reasons articulated previously. However, this poor reliability can cause differential and undue harm to certain defendants.…”

“…Although forensic evaluators are not triers of fact, judicial decisions are overwhelmingly correlated to opinions of forensic evaluators; 2 studies have shown that judges follow the opinions of evaluators in 76%-99% of cases. [3][4][5][6] These opinions can be far-reaching. For example, evaluations of adjudicative competency or sanity can influence whether a defendant is temporarily detained in a correctional facility, hospital, or released to the community-and they may also ultimately lead to charges being dismissed.…”

Competence to stand trial and criminalization: an overview of the research

“…88,90 In a study of competency evaluation reports in Utah where 2 reports were completed on each of 50 defendants, Skeem et al 81 found 82% agreement between evaluators regarding overall competency status, but this dropped to an average rate of about 25% when examining agreement between evaluators on whether a particular psycholegal ability was impaired, with agreement on many of the psycholegal abilities examined at less than 10%. In a large field reliability study, Gowensmith et al 91 examined 216 competency cases in Hawaii, where statute requires 3 separate evaluations be conducted by independent clinicians for each felony defendant referred for competency evaluation. Results indicated that all 3 evaluators arrived at the same opinion regarding competence in 71% of cases.…”