Training Mixed-Domain Translation Models via Federated Learning

Passban, Peyman; Roosta, Tanya; Gupta, Rahul; Chadha, Ankit; Chung, Clement

doi:10.18653/v1/2022.naacl-main.186

Cited by 5 publications

(3 citation statements)

References 1 publication

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Federated Learning in NLP In order to protect data privacy, federated Learning (FL) has attracted a lot of attention from both academic and industrial (Konečnỳ et al, 2016;McMahan et al, 2017;Yang et al, 2019;Kairouz et al, 2021). With more and more language assistance products being applied in real-world applications, FL has also increasingly appeared in the community of NLP to address the problem of privacy leakage, such as machine translation (Passban et al, 2022;Du et al, 2023) and question answering Ait-Mlouk et al, 2022), and so on Cai et al, 2023;.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Tunable Soft Prompts are Messengers in Federated Learning

Dong,

Xie,

Ding

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources, alleviating privacy concerns that arise from directly sharing local data. However, the lack of model privacy protection in FL becomes an unneglectable challenge, especially when people want to federally finetune models based on a proprietary large language model. In this study, we propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts. These soft prompts, updated and transmitted between the server and clients, assume the role of the global model parameters and serve as messengers to deliver useful knowledge from the local data and global model. As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL. Extensive experiments show the effectiveness of the proposed approach compared to several baselines. We have released the source code at https: //github.

show abstract

Section: Related Workmentioning

confidence: 99%

“…(iii) Efficient communication. For example, Passban et al (2022) presents a dynamic pulling FL method to dynamically control the communication bandwidth. Du et al (2023) presents a federated nearest neighbor framework to reduce the communication overhead.…”

Section: Related Workmentioning

confidence: 99%

Tunable Soft Prompts are Messengers in Federated Learning

Dong,

Xie,

Ding

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…Language modeling is one of the fundamental tasks in Natural Language Processing and FL for language modeling recently attracted attention in academia and industry [29,37,6].…”

Section: Related Workmentioning

confidence: 99%

Fingerprint Attack: Client De-Anonymization in Federated Learning

Xu,

Cohn,

Ohrimenko

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Federated Learning allows collaborative training without data sharing in settings where participants do not trust the central server and one another. Privacy can be further improved by ensuring that communication between the participants and the server is anonymized through a shuffle; decoupling the participant identity from their data. This paper seeks to examine whether such a defense is adequate to guarantee anonymity, by proposing a novel fingerprinting attack over gradients sent by the participants to the server. We show that clustering of gradients can easily break the anonymization in an empirical study of learning federated language models on two language corpora. We then show that training with differential privacy can provide a practical defense against our fingerprint attack.

show abstract

Incorporating Collaborative and Active Learning Strategies in the Design and Deployment of a Master Course on Computer-Assisted Scientific Translation

Zappatore

2023

Tech Know Learn

View full text Add to dashboard Cite

This research aims to address the current gaps in computer-assisted translation (CAT) courses offered in bachelor’s and master’s programmes in scientific and technical translation (STT). A multi-framework course design methodology is proposed to support CAT teachers from the computer engineering field, improve student engagement, and promote computer-supported education, together with a balanced coverage of the most relevant topics in the CAT domain. STT is currently in high demand in many fields, requiring translators with sector-specific language skills and considerable computer literacy in order to manage translation projects with complex structures, and format heterogeneity. However, many STT curricula often lag behind current market demands, focusing primarily on language and translation theory, with less emphasis on CAT technologies and tools. Moreover, the lack of shared course design guidelines hinders the introduction of innovative teaching approaches based on collaborative learning. A novel multi-framework CAT course design methodology, named CATDeM, is proposed, based on the integration between an official European translation competence framework, real-life-mimicking laboratorial activities, and computer-supported collaborative learning, enriched with discussion case studies and role-playing experiences. A real-life case study is examined to illustrate and evaluate the implementation of CATDeM in two consecutive editions (2020/2021 and 2021/2022) of a one-semester compulsory CAT course in a M.A. degree in STT at the University of Salento (Italy). Students’ perceptions of translation technology and role-plays, as well as their attitudes towards the proposed CAT course are evaluated through a post-grading self-assessment questionnaire. Achieved results indicated successful student engagement and self-assessed improvement in translation, technical, and interpersonal skills. The importance given by students to role-playing experiences mimicking professional scenarios was also highlighted, paving the way for CATDeM to be adopted in similar contexts.

show abstract

Training Mixed-Domain Translation Models via Federated Learning

Cited by 5 publications

References 1 publication

Tunable Soft Prompts are Messengers in Federated Learning

Tunable Soft Prompts are Messengers in Federated Learning

Fingerprint Attack: Client De-Anonymization in Federated Learning

Incorporating Collaborative and Active Learning Strategies in the Design and Deployment of a Master Course on Computer-Assisted Scientific Translation

Contact Info

Product

Resources

About