Secure and Efficient Federated Transfer Learning

Abstract

Machine Learning models require a vast amount of data for accurate training. In reality, most data is scattered across different organizations and cannot be easily integrated under many legal and practical constraints. Federated Transfer Learning (FTL) was introduced in [1] to improve statistical models under a data federation that allow knowledge to be shared without compromising user privacy, and enable complementary knowledge to be transferred in the network. As a result, a target-domain party can build more flexible and powerful models by leveraging rich labels from a source-domain party. However, the excessive computational overhead of the security protocol involved in this model rendered it impractical. In this work, we aim towards enhancing the efficiency and security of existing models for practical collaborative training under a data federation by incorporating Secret Sharing (SS). In literature, only the semihonest model for Federated Transfer Learning has been considered. In this paper, we improve upon the previous solution, and also allow malicious players who can arbitrarily deviate from the protocol in our FTL model. This is much stronger than the semi-honest model where we assume that parties follow the protocol precisely. We do so using the one of the practical MPC protocol called SPDZ, thus our model can be efficiently extended to any number of parties even in the case of a dishonest majority. In addition, the models evaluated in our setting significantly outperform the previous work, in terms of both runtime and communication cost. A single iteration in our model executes in 0.8 seconds for the semihonest case and 1.4 seconds for the malicious case for 500 samples, as compared to 35 seconds taken by the previous implementation.

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

“…The authors complement the federated transfer learning approach with a secure transfer cross-validation model which employs additively Homomorphic Encryption (HE) to protect the performance of the federated transfer learning approach from the security perspective. In [66], the authors extend and improve upon [65] in two ways (1) reducing the overhead of the security model by an order of magnitude through employing secret sharing instead of homomorphic encryption and (2) extending the semi-honest secure multi-party computation model to consider dishonest malicious parties that might arbitrarily deviate from the federated training process. The SPDZ secure multiparty computation protocol is investigated in this context.…”

A Comprehensive Survey of Privacy-preserving Federated Learning

“…Compared with some secure deep learning methods that suffer from accuracy loss, the FTL can achieve the same accuracy as non-privacy-preserving methods and a higher accuracy than non-federated self-learning methods. Sharma et al [157] proposed a secure and efficient FTL framework based on a multi-party computation. This allows clients to train a transfer learning model while keeping their datasets private against adversaries.…”

“…Secret sharing [156] is a cryptographic technique guaranteeing that a secret consisting of n shares can be reconstructed only when a sufficient number of shares are combined. Secret sharing has been used in many FL frameworks to achieve privacy preservation [38,59,113,126,157,169,198,224]. For example, Bonawitz et al [13] proposed a practical and secure framework for the FL based on the secret sharing.…”

Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues

“…An authentication mechanism is also incorporated to verify the clients. Sharm et al [30] enhance the security of existing federated transfer learning models under malicious setting where some players could arbitrarily deviate from the predefined protocol. They use a variant of Multi-Party Computation (MPC) to improve usability under existence of malicious clients.…”