On the limits of automatic speaker verification: Explaining degraded recognizer scores through acoustic changes resulting from voice disguise

Abstract

In speaker verification research, objective performance benchmarking of listeners and automatic speaker verification (ASV) systems are of key importance in understanding the limits of speaker recognition. While the adoption of common data and metrics has been instrumental to progress in ASV, there are two major shortcomings. First, the utterances lack intentional voice changes imposed by the speaker. Second, the standard evaluation metrics focus on average performance across all speakers and trials. As a result, a knowledge gap remains in how the acoustic changes impact recognition performance at the level of individual speakers. This paper addresses the limits of speaker recognition in ASV systems under voice disguise using a linear mixed effects model to analyze the impact of change in long-term statistics of selected features (formants F1-F4, the bandwidths B1-B4, F0, and speaking rate) to ASV log-likelihood ratio (LLR) score. The correlations between the proposed predictive model and the LLR scores are 0.72 for females and 0.81 for male speakers. As a whole, the difference in long-term F0 between enrollment and test utterances was found to be the individually most detrimental factor, even if the ASV system uses only spectral, rather than prosodic, features. V

Why Did the x-Vector System Miss a Target Speaker? Impact of Acoustic Mismatch Upon Target Score on VoxCeleb Data

“…In the present study, we analyze the speaker variation of the entire VoxCeleb1's dataset, with speech from all the 1251 speakers, 561 female and 690 male, comprising 121,350 and 168,571 same speaker trials respectively. In contrast to our previous study where speakers were asked to disguise their voices [5], the speech variations in the VoxCeleb dataset correspond to the circumstances in which they are performed -whether a live-show interview with an audience, a radio or TV program in a formal or informal atmosphere. [11].…”

“…It is therefore plausible that target speakers may get easily missed on VoxCeleb data, too. Thus, another aim of our work is to address generalizability of our earlier findings [5] (for acted voice data) to contemporary speech present in the VoxCeleb dataset.…”

“…Our work is reminiscent of score calibration [4] where ASV score is adjusted with the aid of external quality signals as side information. Nonetheless, besides using different methodology [5], our perspective is on explanatory analysis of a ASV system on a given evaluation corpus, rather than on improving predictive performance. Other related research includes probing information in speaker embeddings [6,7].…”

“…We extend upon our recent work [5] in terms of speech database size, qualities, and the selected acoustic features. In [5] we used a self-collected (now publicly available) AVOID corpus of 60 Finnish speakers.…”

“…We extend upon our recent work [5] in terms of speech database size, qualities, and the selected acoustic features. In [5] we used a self-collected (now publicly available) AVOID corpus of 60 Finnish speakers. The speakers were asked to purposefully modify their voices to sound like old and child speakers, so as to purposefully reinforce large variation between enrollment and test data.…”