Configurable Privacy-Preserving Automatic Speech Recognition

Aloufi, Ranya; Haddadi, Hamed; Boyle, David

doi:10.21437/interspeech.2021-1783

Cited by 5 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an alternative solution, we utilized the wav2vec2 model to produce the log probabilities for all possible characters and then used a convolutional recurrent deep neural network (CRDNN) to transform them into categorical labels (W2V2 logp). These models combine the advantages of convolutional and recurrent neurons and became a popular choice in paralinguistics and ASR [21]. To showcase the benefits of using the wav2vec2 model, we also trained CRDNNs that classify the examples based on their MFCC features (MFCC+CRDNN).…”

Section: Speech Rating Systemmentioning

confidence: 99%

wav2vec2-based Speech Rating System for Children with Speech Sound Disorder

Getman¹,

Al-Ghezi²,

Voskoboinik³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Speaking is a fundamental way of communication, developed at a young age. Unfortunately, some children with speech sound disorder struggle to acquire this skill, hindering their ability to communicate efficiently. Speech therapies, which could aid these children in speech acquisition, greatly rely on speech practice trials and accurate feedback about their pronunciations. To enable home therapy and lessen the burden on speech-language pathologists, we need a highly accurate and automatic way of assessing the quality of speech uttered by young children. Our work focuses on exploring the applicability of state-of-the-art self-supervised, deep acoustic models, mainly wav2vec2, for this task. The empirical results highlight that these self-supervised models are superior to traditional approaches and close the gap between machine and human performance.

show abstract

Section: Speech Rating Systemmentioning

confidence: 99%

wav2vec2-based Speech Rating System for Children with Speech Sound Disorder

Getman¹,

Al-Ghezi²,

Voskoboinik³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

show abstract

“…It is very likely that the biometric data obtained in interaction with online services will be utilized to extract a fair amount of personal characteristics, and information about the circumstances and environment of users from their raw data. For example, an overlearning problem might be caused by deep neural networks used for ASR and speaker verification, revealing additional/sensitive information about the users, which threatens to compromise their privacy significantly [16,92].…”

Section: Threats To User Privacymentioning

confidence: 99%

“…Using our pipeline, we recommend validating the data source at the edge before sharing it, see Figure 1. We plan to integrate new applications [14,16,122] with a configurable privacy engine and evaluate the effectiveness of source validation in mitigating spoofing attempts while retaining its usefulness in minimizing privacy intrusions.…”

Section: Future Work and Open Directions 61 Beyond Authenticationmentioning

confidence: 99%

Locally Authenticated Privacy-preserving Voice Input

Aloufi¹,

Nautsch²,

Haddadi³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Increasing use of our biometrics (e.g., fingerprints, faces, or voices) to unlock access to and interact with online services raises concerns about the trade-offs between convenience, privacy, and security. Service providers must authenticate their users, although individuals may wish to maintain privacy and limit the disclosure of sensitive attributes beyond the authentication step, e.g., when interacting with Voice User Interfaces (VUIs). Preserving privacy while performing authentication is challenging, particularly where adversaries can use biometric data to train transformation tools (e.g., 'deepfaked' speech) and use the faked output to defeat existing authentication systems. In this paper, we take a step towards understanding security and privacy requirements to establish the threat and defense boundaries. We introduce a secure, flexible privacypreserving system to capture and store an on-device fingerprint of the users' raw signals (i.e., voice) for authentication instead of sending/sharing the raw biometric signals. We then analyze this fingerprint using different predictors, each evaluating its legitimacy from a different perspective (e.g., target identity claim, spoofing attempt, and liveness). We fuse multiple predictors' decisions to make a final decision on whether the user input is legitimate or not. Validating legitimate users yields an accuracy rate of 98.68% after cross-validation using our verification technique. The pipeline runs in tens of milliseconds when tested on a CPU and a single-core ARM processor, without specialized hardware.

show abstract

“…Voice-controlled IoT devices and smart home assistants are gaining popularity in people's lives, and spoken language understanding (SLU) has become one of the main enabling technologies to achieve human-machine interaction between users and devices [Coucke et al, 2018]. For example, SLU can be found in the in-car voice system of Tesla electricity vehicles (EVs), the voice assistant of Apple phones and Xiaomi's smart home ecology.…”

Section: Introductionmentioning

confidence: 99%

“…There are certain privacy risks associated with these IOT devices [Atlam and Wills, 2020]. Due to their limited storage space, some vendors choose to deploy only the encoder of the model at the client side and transmit the output of the encoder to the server side for performing tasks such as prediction and classification, which gives opportunities for malicious attackers to take advantage of.…”

Section: Introductionmentioning

confidence: 99%

Privacy-Preserving End-to-End Spoken Language Understanding

Wang

Huang

Yang

2023

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Spoken language understanding (SLU), one of the key enabling technologies for human-computer interaction in IoT devices, provides an easy-to-use user interface. Human speech can contain a lot of user-sensitive information, such as gender, identity, and sensitive content. New types of security and privacy breaches have thus emerged. Users do not want to expose their personal sensitive information to malicious attacks by untrusted third parties. Thus, the SLU system needs to ensure that a potential malicious attacker cannot deduce the sensitive attributes of the users, while it should avoid greatly compromising the SLU accuracy. To address the above challenge, this paper proposes a novel SLU multi-task privacy-preserving model to prevent both the speech recognition (ASR) and identity recognition (IR) attacks. The model uses the hidden layer separation technique so that SLU information is distributed only in a specific portion of the hidden layer, and the other two types of information are removed to obtain a privacy-secure hidden layer. In order to achieve good balance between efficiency and privacy, we introduce a new mechanism of model pre-training, namely joint adversarial training, to further enhance the user privacy. Experiments over two SLU datasets show that the proposed method can reduce the accuracy of both the ASR and IR attacks close to that of a random guess, while leaving the SLU performance largely unaffected.

show abstract

Configurable Privacy-Preserving Automatic Speech Recognition

Cited by 5 publications

References 18 publications

wav2vec2-based Speech Rating System for Children with Speech Sound Disorder

wav2vec2-based Speech Rating System for Children with Speech Sound Disorder

Locally Authenticated Privacy-preserving Voice Input

Privacy-Preserving End-to-End Spoken Language Understanding

Contact Info

Product

Resources

About