Accuracy, reproducibility, and repeatability of forensic footwear examiner decisions

Do inconclusive forensic decisions disadvantage the innocent?

“…The tendency for the experts in these studies to disproportionately apply inconclusive decisions to different-source evidence, although not universal (it does not seem to characterize DNA evidence, for instance; Butler et al, 2018), is not an anomaly. Since publication of the PCAST report, other forensic validation studies that also met all or most of the PCAST’s criteria for scientific rigor have replicated the pattern for cartridge cases (Guyll et al, 2023; Monson et al, 2023; Neuman et al, 2022) and shown it to also characterize bullets, footwear, handwriting, and palm print evidence (Eldridge et al, 2021; Hicklin, Eisenhart, et al, 2022; Hicklin, McVicker, et al, 2022; Monson et al, 2023; Neuman et al, 2022). Moreover, an analytic review of the polygraph showed an analogous pattern whereby polygraphers applied inconclusive decisions to truthful suspects nearly three times more often than to deceptive suspects (Honts & Schweinle, 2009).…”

“…Forensic validation studies indicate that inconclusive decisions have exculpatory probative value (Baldwin et al, 2023; Eldridge et al, 2021; Guyll et al, 2023; Hicklin, Eisenhart, et al, 2022; Hicklin, McVicker, et al, 2022; Monson et al, 2023; Neuman et al, 2022; Pacheco et al, 2014). Yet Experiment 1 showed that the inconclusive decision did not produce clearly exonerating judgments, supporting the hypothesis under investigation.…”

“…Inconclusive forensic decisions predict different-source evidence status more strongly than same-source evidence status and, thus, possess exculpatory probative value (Baldwin et al, 2023; Eldridge et al, 2021; Guyll et al, 2023; Hicklin, Eisenhart, et al, 2022; Hicklin, McVicker, et al, 2022; Monson et al, 2023; Neuman et al, 2022; Pacheco et al, 2014). However, participants in Experiment 2 interpreted the inconclusive decision to be uninformative with regard to a suspect’s guilt status.…”

“…Rigorous forensic validation studies pertaining to the forensic analysis of cartridge cases, bullets, latent prints, handwriting, and footwear evidence demonstrate that inconclusive decisions empirically predict different-source evidence status more strongly than same-source evidence status (Baldwin et al, 2023; Eldridge et al, 2021; Guyll et al, 2023; Hicklin, Eisenhart, et al, 2022; Hicklin, McVicker, et al, 2022; Monson et al, 2023; Neuman et al, 2022; Pacheco et al, 2014). However, this is not how participants in our research interpreted a forensic expert’s inconclusive decision.…”

An automated alignment algorithm for identification of the source of footwear impressions with common class characteristics

“…Examiners vary not only in their experience but also in attributes such as visual memory and perceptual skills that may result in different conclusions when more than one individual examines the same items. A recent black box study that included over 80 forensic footwear examiners [2] suggests that the frequency of false identifications and false exclusions tends to be low. However, reproducibility (or inter-rater consistency) and repeatability (intra-rater consistency) were also low.…”