Private Cross-Silo Federated Learning for Extracting Vaccine Adverse Event Mentions

Abstract

Federated Learning (FL) is quickly becoming a goto distributed training paradigm for users to jointly train a global model without physically sharing their data. Users can indirectly contribute to, and directly benefit from a much larger aggregate data corpus used to train the global model. However, literature on successful application of FL in real-world problem settings is somewhat sparse. In this paper, we describe our experience applying a FL based solution to the Named Entity Recognition (NER) task for an adverse event detection application in the context of mass scale vaccination programs. We present a comprehensive empirical analysis of various dimensions of benefits gained with FL based training. Furthermore, we investigate effects of tighter Differential Privacy (DP) constraints in highly sensitive settings where federation users must enforce Local DP to ensure strict privacy guarantees. We show that local DP can severely cripple the global model's prediction accuracy, thus disincentivizing users from participating in the federation. In response, we demonstrate how recent innovation on personalization methods can help significantly recover the lost accuracy. We focus our analysis on the Federated Fine-Tuning algorithm, FedFT, and prove that it is not PAC Identifiable, thus making it even more attractive for FL-based training.

Adoption of Federated Learning for Healthcare Informatics: Emerging Applications and Future Directions

“…Improvement in precision of the model is not mentioned Chamikaraa et al [95] 2021 Proposed an algorithm to control the global perturbation parameter generation, whereas local data perturbation can be conducted by the distributed entities Higher privacy and accuracy is achieved Higher efficiency with vertical FL is not mentioned Kanani et al [96] 2021 Proposed an FL based solution for adverse event detection for mass scale vaccination programs.…”

“…Comparative analysis of the proposed algorithm with the existing schemes based on the classification accuracy, time complexity, and attack resistance is also mentioned. Further, Kanani et al [96] proposed a real-world application to detect adverse events in mass-scale vaccination programs. Detailed analysis of Federated fine-tuning algorithm with better performance and DP is mentioned.…”

FLightNER: A Federated Learning Approach to Lightweight Named-Entity Recognition

“…For example, the smaller version of BERT contains 110 million parameters [5]. Finally, in FL setting, Kanani et al [21] showed that fine-tuning does not improve prediction performance for all users, and in some cases, performance degrades.…”

“…NER applicability extends beyond medical reports and includes multiple domains such as social media, news organizations and text anonymization. Traditionally, such named entities were recognized and annotated by human experts, which is an expensive and timeconsuming task [21]. Using FL, several small health organizations can participate in producing a global model that would otherwise have not been possible given their insufficient labelled data [21].…”

“…Traditionally, such named entities were recognized and annotated by human experts, which is an expensive and timeconsuming task [21]. Using FL, several small health organizations can participate in producing a global model that would otherwise have not been possible given their insufficient labelled data [21]. Likewise, large departments within a single organization can train a global model to achieve a common machine learning objective without moving their data across jurisdictions and comply with data sharing regulations [17].…”