Collective intelligence in fingerprint analysis

Abstract

When a fingerprint is located at a crime scene, a human examiner is counted upon to manually compare this print to those stored in a database. Several experiments have now shown that these professional analysts are highly accurate, but not infallible, much like other fields that involve high-stakes decision-making. One method to offset mistakes in these safety-critical domains is to distribute these important decisions to groups of raters who independently assess the same information. This redundancy in the system allows it to continue operating effectively even in the face of rare and random errors. Here, we extend this "wisdom of crowds" approach to fingerprint analysis by comparing the performance of individuals to crowds of professional analysts. We replicate the previous findings that individual experts greatly outperform individual novices, particularly in their false-positive rate, but they do make mistakes. When we pool the decisions of small groups of experts by selecting the decision of the majority, however, their false-positive rate decreases by up to 8% and their false-negative rate decreases by up to 12%. Pooling the decisions of novices results in a similar drop in false negatives, but increases their falsepositive rate by up to 11%. Aggregating people's judgements by selecting the majority decision performs better than selecting the decision of the most confident or the most experienced rater. Our results show that combining independent judgements from small groups of fingerprint analysts can improve their performance and prevent these mistakes from entering courts.

Pooling decisions decreases variation in response bias and accuracy

“…For example, the MPAD in sensitivity between two randomly selected single experts is 0.13 for breast cancer, 0.13 for skin cancer, and 0.17 for fingerprint recognition; pooling the decisions of five experts reduces these values to 0.06, 0.06, and 0.10, which amounts to relative reductions of 52%, 54%, and 41%, respectively. Of importance, and as reported before (Kurvers et al, 2015;Tangen et al, 2020;Wolf et al, 2015) 4C and 4D) while increasing mean accuracy (Figures 4E and 4F). Figure S6 shows that similar results emerge when A-C).…”

“…(3) a fingerprint recognition dataset, comprising 1,584 evaluations by 36 professional fingerprint examiners of 44 fingerprint pairs (Tangen et al, 2020), (4) a geopolitical forecasting dataset from the Good Judgment Project, containing 8,258 forecasts by 89 forecasters on 94 geopolitical events (Ungar et al, 2012); and (5) a general knowledge dataset, containing 99,000 responses by 99 individuals to 1,000 questions (here: which of two cities is larger) (Yu et al, 2015). All datasets are described in detail in the STAR methods.…”

Pooling decisions decreases variation in response bias and accuracy

“…For example, the MAD in sensitivity between two randomly selected single experts is 0.125 for breast cancer, 0.13 for skin cancer, and 0.17 for fingerprint recognition; pooling the decisions of five experts reduces these values to 0.06, 0.06, and 0.10-relative reductions of 52%, 54% and 41%, respectively. Importantly, and as reported before(14,30,33) mean sensitivity(Fig. 3E, G, I) and specificity(Fig.…”

“…The full information on the fingerprint analyses dataset can be found in (30), therefore, we provide a brief On the final four trials (2 targets, 2 distractors), they had an unlimited amount of time to make a decision.…”

A Gamified Simulator and Physical Platform for Self-Driving Algorithm Training and Validation

“…To meet the needs of robust, diverse and high quality data reflecting safe and ethical driving, the capture of known-good and large-scale data is necessary. "Wisdom of the crowd" requires massive scale, particularly in critical systems [4], so companies may instead capture data from costly trained drivers to maximize quality at the expense of quantity.…”