Adversarial Disentanglement of Speaker Representation for Attribute-Driven Privacy Preservation

Abstract

In speech technologies, speaker's voice representation is used in many applications such as speech recognition, voice conversion, speech synthesis and, obviously, user authentication. Modern vocal representations of the speaker are based on neural embeddings. In addition to the targeted information, these representations usually contain sensitive information about the speaker, like the age, sex, physical state, education level or ethnicity. In order to allow the user to choose which information to protect, we introduce in this paper the concept of attributedriven privacy preservation in speaker voice representation. It allows a person to hide one or more personal aspects to a potential malicious interceptor and to the application provider. As a first solution to this concept, we propose to use an adversarial autoencoding method that disentangles in the voice representation a given speaker attribute thus allowing its concealment. We focus here on the sex attribute for an Automatic Speaker Verification (ASV) task. Experiments carried out using the VoxCeleb datasets have shown that the proposed method enables the concealment of this attribute while preserving ASV ability.

Investigating the contribution of speaker attributes to speaker separability using disentangled speaker representations

“…SplitDim-Adv models were only successful when training from scratch or by initializing using another SplitDim-Adv model (as was the case when fine-tuning SplitDim-Adv from Vox-Celeb to SCOTUS). This could explain the findings of [21,20], In Table 2, the speaker verification performance on Vox-Celeb is shown for the baseline model, along with the SplitDim-Adv model. Firstly, we can see that disentangling the space has incurred a reduction in performance (4.22% to 6.68% EER), which is likely due to the addition of the four extra tasks of the SplitDim-Adv model (Gender, Gender-Adversary, Nationality, Nationality-Adversary).…”

“…The topic of disentangled speaker representations is also closely linked with the field of voice privacy [17,18,19], wherein certain attributes are desirable to obscure in speaker embeddings to protect against malicious attackers. Notably, the work of [20] used adversarial training to control the gender element of an auto-encoder architecture, seeking to be able to control that element and therefore provide gender-invariant representations. A follow up paper [21] utilized normalizing flows to again obscure the gender information in speaker embeddings, finding this to be an improvement over the adversarial method.…”

Adversarial Representation Learning for Robust Privacy Preservation in Audio

“…Later studies focused on removing specific speaker attributes instead of general speaker identity information. For example, Noé et al [18] used adversarial training to preserve the privacy of speakers' gender information in an automatic speaker verification system. While this study investigated a single attribute, in practice, there may be a need to conceal multiple types of information.…”

Generating gender-ambiguous voices for privacy-preserving speech recognition

“…Gender information is typically used to condition models preserving the identity of a speaker. However, only a handful of methods explicitly consider gender as a sensitive attribute to protect [7,8,9,10,11]. A hybrid model combining Variational Autoencoders and Generative Adversarial Networks (GANs) can be used to protect gender information through voice conversion with a disentanglement approach targeted for the speech recognition task [7].…”

“…Two encoders are trained to independently encode content and speaker identity information that is then used to hide (or mask) gender information. Privacy methods that operate at feature-level have been used to disentangle gender information from x-vectors [12] with adversarial learning and an encoder-decoder based architecture [8]. Because this adversarial method removes the unwanted information at the level of the feature representation instead of the speech waveform, it is not useful for tasks such as speech recognition.…”