In order to assess the extent of agreement between forensic footwear examiners in the United States, a reliability study was performed by West Virginia University between February 2017 and August 2018. Over the span of 19 months, 70 examiners each performed 12 comparisons and reported a total of 840 conclusions. For each comparison, participants were queried on a number of factors in order to determine the degree to which different types of features were identified, evaluated, and weighted, before arriving at a final decision regarding the strength of the association or disassociation between questioned and test impressions. Preliminary results from this study are divided into a series of three summaries. This manuscript (Part I) describes participant demographics as well as community agreement in both feature identification/annotation, and final reporting. Results indicate considerable variation in feature identification/annotation (as low as 66.5% agreement), but higher consistency in the reporting of overall conclusions. For mated pairs, this agreement was 79.7% AE 14.1% (median of 85.7% and a 90% confidence interval between 75.9% and 83.2%). For nonmated pairs, the equivalent overall agreement was 89.8% AE 6.69% (median of 91.4% and a 90% confidence interval between 87.4% and 92.1%). These estimates of agreement are further compared with previous published findings, and collectively, the work extends the body of knowledge concerning reliability in forensic footwear comparisons and conclusions.

mance among forensic footwear examiners in the United States. Throughout the study's duration, 70 examiners each performed 12 comparisons and reported a total of 840 conclusions. In order to assess the accuracy of conclusions, the similarities and differences between mated and nonmated pairs were evaluated according to three criteria: (i) inherent agreement/disagreement in class, wear, and randomly acquired features, (ii) limitations as a function of questioned impression quality, clarity, and totality, and (iii) adherence to the Scientific Working Group for Shoeprint and Tire Tread Evidence (SWGTREAD) 2013 conclusion standard. Using these criteria, acceptable/expected categorical conclusions were defined. Preliminary results from this study are divided into a series of three summaries. This manuscript (Part II) reports accuracy and reproducibility. For mated pairs, accuracy equals 76.3% AE 13.0% (median of 78.6% and a 90% confidence interval between 72.2% and 80.0%). For nonmated pairs, accuracy equals 87.4% AE 9.24% (median of 91.4% and a 90% confidence interval between 84.7% and 89.8%). In addition, the community assessed agreement (denoted by IQR) of reported results equals the research team's accepted/expected conclusions for 10 out of 12 comparisons. In terms of reproducibility, the 90% confidence interval for consensus was computed and found to equal 0.71-0.86 (median of 0.77) for the combined dataset. Although based on a limited sample size, these results provide a baseline estimate of accuracy and consensus/ reproducibility as a function of the existing seven-point SWGTREAD 2013 conclusion standard.

Over the course of 19 months, West Virginia University collected reports from 70 footwear experts, each performing 12 questioned-test comparisons, resulting in a dataset that includes more than 1000 examiner attributes (education, training, certification status, etc.), 3500 impression features identified and evaluated (clarity, totality, and similarity), and 840 source conclusions. The results were used to estimate the performance of forensic footwear examiners in the United States, including error rates, predictive value (PV), and measures of interrater reliability (IRR). For the dataset and mate-prevalence (31.5%) used in this study, results indicate correct predictive value varies from 94.5% for exclusions, 85.0% for identifications, and between 70.1% and 65.2% for limited associations and association of class, respectively (with all other conclusions producing PVs between these extremes). After data transformation based on ground truth, the case study materials show a false-positive rate of 0.48%, a false-negative rate of 15.6%, a (correct) positive predictive value of 98.8%, and a (correct) negative predictive value of 93.3%. In addition to error rates and PVs, inter-rater reliability was likewise computed to describe examiner reproducibility; results indicate a Gwet AC 2 agreement coefficient of 0.751-0.692 when using a six-and four-level reporting structure, respectively, which translates into "substantial" and "moderate agreement" for a benchmarked verbal equivalent scale. The reported performance metrics are further compared against past forensic footwear reliability studies, including a discussion of how the use of a six-level reporting structure impacts results.

No abstract

Barefoot impression evidence may be used to link a person to a shoe or foot impression found at a crime scene. The analysis of barefoot impressions involves the examination and comparison of the morphology and test impressions of the foot, to the questioned crime scene impression. Although a lesser known type of physical evidence, barefoot evidence has proven to be a valuable aid in criminal casework.