Voice biometrics security: Extrapolating false alarm rate via hierarchical Bayesian modeling of speaker verification scores

Abstract

How secure automatic speaker verification (ASV) technology is? More concretely, given a specific target speaker, how likely is it to find another person who gets falsely accepted as that target? This question may be addressed empirically by studying naturally confusable pairs of speakers within a large enough corpus. To this end, one might expect to find at least some speaker pairs that are indistinguishable from each other in terms of ASV. To a certain extent, such aim is mirrored in the standardized ASV evaluation benchmarks, for instance, the series of speaker recognition evaluation (SRE) organized by the National Institute of Standards and Technology (NIST). Nonetheless, arguably the number of speakers in such evaluation benchmarks represents only a small fraction of all possible human voices, making it challenging to extrapolate performance beyond a given corpus. Furthermore, the impostors used in performance evaluation are usually selected randomly. A potentially more meaningful definition of an impostor -at least in the context of security-driven ASV applications -would be closest (most confusable) other speaker to a given target.We put forward a novel performance assessment framework to address both the inadequacy of the random-impostor evaluation model and the size limitation of evaluation corpora by addressing ASV security against closest impostors on arbitrarily large datasets. The framework allows one to make a prediction of the safety of given ASV technology, in its current state, for arbitrarily large speaker database size consisting of virtual (sampled) speakers. As a proof-of-concept, we analyze the performance of two state-of-the-art ASV systems, based on i-vector and x-vector speaker embeddings (as implemented in the popular Kaldi toolkit), on the recent VoxCeleb 1 & 2 corpora, containing a total of 7,365 speakers. We fix the number of target speakers to 1000, and generate up to N = 100, 000 virtual impostors sampled from the generative model. The model-based false alarm rates are in a reasonable agreement with empirical false alarm rates and, as predicted, increase substantially (values up to 98%) with N = 100, 000 impostors. Neither the i-vector or x-vector system is immune to increased false alarm rate at increased impostor database size, as predicted by the model.

Extrapolating False Alarm Rates in Automatic Speaker Verification

“…Next, use the generated sets of scores to estimate P N FA (τ ) using ( 4). We used this approach in [3] where a location-scale model with Gaussian base distribution was trained to maximize model log-likelihood [17]. Different from [3], the models proposed in this work are instances of implicit generative models: they are specified through a forward stochastic procedure for data generation, but do not allow direct likelihood evaluation [23,24].…”

“…The first set was used to train the ASV systems. The second one is the standard Voxceleb1 evaluation protocol [32], used as a sanity check of our ASV systems (see [3] for details). The third set which contains 1000 male and 1000 female speakers was used to compute scores for training models for P N FA extrapolation.…”

“…It also assumes that the closest impostor has the largest location parameter µj. One limitation of the model in [3] is the unrealistic assumption of Gaussian betweenspeaker scores. Here F (•) is allowed to be arbitrary.…”

A broad review on non-intrusive active user authentication in biometrics

“…Voice has also been investigated with voice mimicry attack (Vestman et al 2020) and found to be vulnerable. Voice biometric security is normally tested against general datasets with less percentage of similar signature imposter data (Shirvanian et al 2019;Sholokhov et al 2020). The testing of ASV systems needs to incorporate the closest imposter model framework for more accurate results (Lakshmi et al 2020) with the use of chaotic maps.…”

A Method for the Real-Time Updating of Voice Samples in the Unified Biometric System

“…As one of the most urgent problems in this fi eld, we can mention the necessity of regular updating of accumulated BPD samples [6,7]. It seems to be especially important in connection with recording of voice samples [8][9][10]. Due to the phenomenon of variability of speech signal on the sound (phonetic) level of its perception, any voice samples inevitably ages with time and gradually loses its consumer quality [11,12].…”