Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings

Abstract

Speech emotion recognition (SER) processes speech signals to detect and characterize expressed perceived emotions. Many SER application systems often acquire and transmit speech data collected at the client-side to remote cloud platforms for inference and decision making. However, speech data carry rich information not only about emotions conveyed in vocal expressions, but also other sensitive demographic traits such as gender, age and language background. Consequently, it is desirable for SER systems to have the ability to classify emotion constructs while preventing unintended/improper inferences of sensitive and demographic information. Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing their local data. This training approach appears secure and can improve privacy for SER. However, recent works have demonstrated that FL approaches are still vulnerable to various privacy attacks like reconstruction attacks and membership inference attacks. Although most of these have focused on computer vision applications, such information leakages exist in the SER systems trained using the FL technique. To assess the information leakage of SER systems trained using FL, we propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters, corresponding to the FedSGD and the FedAvg training algorithms, respectively. As a use case, we empirically evaluate our approach for predicting the client's gender information using three SER benchmark datasets: IEMOCAP, CREMA-D, and MSP-Improv. We show that the attribute inference attack is achievable for SER systems trained using FL. We further identify that most information leakage possibly comes from the first layer in the SER model.

User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learning

“…For instance, many works have demonstrated that data reconstruction is achievable through analyzing the model updates in FL setup [8,9,10]. We had previously demonstrated this phenomenon in FL-based SER setup [6]. Specifically, we showed that an attribute inference attacker could successfully infer a user's gender attribute by using the model updates shared in the FL setup [6].…”

“…In this section, we first review the attacking framework we proposed in [6]. We then summarise the proposed UDP algorithm used in this work.…”

“…We had previously demonstrated this phenomenon in FL-based SER setup [6]. Specifically, we showed that an attribute inference attacker could successfully infer a user's gender attribute by using the model updates shared in the FL setup [6]. A typical approach to protect privacy in FL is differential privacy (DP) [11,12], of which local DP (LDP) is a prominent example [7].…”

“…Federated Learning (FL) methods attempt to address the issues of data privacy by training a model on a central server using the shared model parameters from an edge device without the need for local data [5]. However, as reported in our prior work, SER applications trained in an FL setup are still vulnerable to attribute inference attacks [6]. In particular, we found that an adversary with access to local parameter updates can successfully infer the gender of the user (deemed as sensitive in that particular SER use case) operating the edge device.…”

Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling

“…return θ, c ′ , ∇c prior work [25]. Finally, we calculate the global average of the last layer's hidden state as the final feature from the pre-trained model's output.…”