Telling us less than what they know: Expert inconclusive reports conceal exculpatory evidence in forensic cartridge-case comparisons.

Abstract

Cartridge-case comparison experts are tasked with determining whether a reference sample associated with the suspect originated from the same source as the evidence sample found at the crime scene. Same-source reports have strong inculpatory value and different-source reports have strong exculpatory value. Surprisingly, inconclusive reports have strong exculpatory value as they occur much more frequently for actual nonmatches than for actual matches. We used a signal detection model to test our hypothesis that some experts report inconclusive when they detect a nonmatch. Consistent with this hypothesis, we found that even when examiners were able to perfectly discriminate matches from nonmatches, they rendered inconclusive reports on 32% of nonmatch trials. Experts are biased to avoid rendering different-source reports, which conceals exculpatory information that innocent persons desperately need to establish their innocence. We argue that this biased examiner might result from an adversarial allegiance bias combined with a flawed response scale.

Implicit Blackstone ratios in decisions made by firearm and toolmark examiners.

“…Smith and Wells (2024) reported that the identification thresholds (C4) are PSM = 2.4 for Berretta cartridge case comparisons and PSM = 2.3 for HiPoint cartridge case comparisons. The empirical LRs for Berretta and HiPoint density ratios at these thresholds are 9.41 (=0.2107/0.0224) and 0.81 (=0.0229/0.0283), respectively.…”

“…All three components affect the placement of a decision threshold (DeKay, 1996). The analysis by Smith and Wells (2024) elucidates the discriminability of firearm examiners. In this article, we extend their findings to examine the Blackstone ratios implicitly used by firearm examiners.…”

“…Signal detection theory posits that a rational decision made under uncertainty is a function of three properties: discriminability, base rates, and the relative cost of false positive and false negative errors, referred to in legal contexts as a Blackstone ratio . Building on the analysis by Smith and Wells (2024), which focuses on discriminability, we use an algebraic approach to calculate the Blackstone ratios implicitly used by firearm examiners. Contrary to claims that examiners are extremely averse to false positive errors, the implied Blackstone ratios reflect a modest aversion to false positives for Barretta cartridge case comparisons and a slight preference for false negatives relative to false positives for HiPoint cartridge comparisons.…”

Cartridge-case examiners’ aversion to true rejections: A shocking problem with use of the “inconclusive” category.

“…

We used signal detection modeling of a large data set to show that forensic cartridge-case examiners were highly unlikely to commit false positives or false negatives but appear to be strongly biased against true negatives (Smith & Wells, 2024). Specifically, forensic cartridge-case examiners were far more likely to claim that the results were inconclusive when the shell casings did not match than to say the results were inconclusive when the shell casings matched.

…”

“…Our original article was focused on this asymmetric use of the inconclusive category in which examiners were far more likely to claim the result was inconclusive for actual nonmatches than for actual matches (Smith & Wells, 2024). We fit signal detection models to the Guyll et al (2023) data and concluded that some examiners were calling nonmatches inconclusive despite almost certainly knowing they were nonmatches.…”