Likelihood Ratio Calibration in a Transparent and Testable Forensic Speaker Recognition Framework

Abstract

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: AbstractA recently reopened debate about the infallibility of some classical forensic disciplines is leading to new requirements in forensic science. Standardization of procedures, proficiency testing, transparency in the scientific evaluation of the evidence and testability of the system and protocols are emphasized in order to guarantee the scientific objectivity of the procedures. Those ideas will be exploited in this paper in order to walk towards an appropriate framework for the use of forensic speaker recognition in courts. Evidence is interpreted using the Bayesian approach for the analysis of the evidence, as a scientific and logical methodology, in a two-stage approach based in the similarity-typicality pair, which facilitates the transparency in the process. The concept of calibration as a way of reporting reliable and accurate opinions is also deeply addressed, presenting experimental results which illustrate its effects. The testability of the system is then accomplished by the use of the NIST SRE 2005 evaluation protocol. Recently proposed applicationindependent evaluation techniques (C llr and APE curves) are finally addressed as a proper way for presenting results of proficiency testing in courts, as these evaluation metrics clearly show the influence of calibration errors in the accuracy of the inferential decision process.

Suspect-Adapted MAP Estimation of Within-Source Distributions in Generative Likelihood Ratio Estimation

“…This lack of calibration [19,9], which is not observable in a DET plot, represents a critical issue in forensic speaker recognition systems. This important idea is out of the scope of this paper and is deeply addressed in [12].…”

“…Then the information provided by the system is suboptimal. This effect is solved by the technique proposed below, and is discussed in depth in another recent work from the authors [12]. .…”

“…Moreover, ¥ ¦ ¦ § includes two different measures: i) the loss in accuracy because of the discrimination capabilities of the system (i. e., a bad refinement [19]) and ii) a penalty to ¢ £ values which would lead to unreliable or misleading confidences (which is known as a lack of calibration [19,9]). The reader may consult [9] for a detailed description of the effects of calibration in automatic speaker recognition systems and [12] for its application to forensic speaker recognition.…”

Impact of Background Noise in Mobile Phone Networks on Forensic Voice Comparison

“…The Background set remained the same across all comparisons and included tokens from two recording sessions for each of the 44 Background speakers. The sole purpose of the Development set is to train the fusion system, the resulting weights of which are used to combine LRs calculated from individual vowels for every comparison in the testing set [35].…”

An investigation of supervector regression for forensic voice comparison on small data

“…Tippett plots, which provide a graphical representation of the cumulative distribution function of log-likelihood ratios for same-origin and different-origin hypotheses [9,61], were also used in current study.…”