FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features

Liu, Yang; Zhang, Xinwei; Kang, Yan; Li, Liping; Chen, Tianjian; Mei, Hong; Yang, Qiang

doi:10.1109/tsp.2022.3198176

Cited by 47 publications

(25 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First of all, in the cross-party SSL step, each party only needs the intermediate outputs from the other party to perform local training and doesn't have to wait for the gradients like in naive VFL, thus the SSL can be done asynchronously which can greatly reduce the waiting and scheduling time due to the variance of the linking quality and difference of the computational power. Secondly, to further take advantage of the transferred representation intermediates and the capability of asynchronous training, in FedCSSL each party can perform multiple times of updates with the currently received intermediate (Liu et al 2019). This multi-updates strategy can be integrated with the asynchronous training into an adaptive scheduling system and further reduce the communication cost.…”

Section: Communication Efficiency Analysismentioning

confidence: 99%

A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning

He¹,

Ke²,

Luo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Federated learning (FL) enables independent parties to collaboratively build machine learning (ML) models while protecting data privacy. Vertical federated learning (VFL), a variant of FL, has recently drawn increasing attention as the VFL matches the enterprises' demands of leveraging more valuable features to achieve better model performance without jeopardizing data privacy. However, conventional VFL may run into data deficiency as it is only able to exploit aligned samples (belonging to different parties) with labels, leaving often the majority of unaligned and unlabeled samples unused. The data deficiency hampers the effort of the federation. In this work, we propose a Federated Hybrid Self-Supervised Learning framework, coined FedHSSL, to utilize all available data (including unaligned and unlabeled samples) of participants to train the joint VFL model. The core idea of FedHSSL is to utilize cross-party views (i.e., dispersed features) of samples aligned among parties and local views (i.e., augmentations) of samples within each party to improve the representation learning capability of the joint VFL model through SSL (e.g., SimSiam). FedHSSL further exploits generic features shared among parties to boost the performance of the joint model through partial model aggregation. We empirically demonstrate that our FedHSSL achieves significant performance gains compared with baseline methods, especially when the number of labeled samples is small. We provide an in-depth analysis of FedHSSL regarding privacy leakage, which is rarely discussed in existing self-supervised VFL works. We investigate the protection mechanism for FedHSSL. The results show our protection can thwart the state-of-the-art label inference attack.

show abstract

Section: Communication Efficiency Analysismentioning

confidence: 99%

A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning

He¹,

Ke²,

Luo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Homomorphic encryption allows direct operations on the ciphertext without decryption, which has attracted a large number of researchers to explore and study in recent years [2], [7], [21], [39]. The most commonly used homomorphic encryption algorithm is Paillier [27].…”

Section: Encryption Mechanismmentioning

confidence: 99%

“…However, most FL research focuses on the sample partitioned scenario [32], [37], where all cooperators share the same feature dimension. The feature partitioned federated learning is rarely explored in literature, which is an equally important issue in real industrial scenarios, such as recommender system [13], credit evaluation [21], etc.…”

Section: Introductionmentioning

confidence: 99%

Accelerating Vertical Federated Learning

Cai,

Fan,

Kang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Privacy, security and data governance constraints rule out a brute force process in the integration of cross-silo data, which inherits the development of the Internet of Things. Federated learning is proposed to ensure that all parties can collaboratively complete the training task while the data is not out of the local. Vertical federated learning is a specialization of federated learning for distributed features. To preserve privacy, homomorphic encryption is applied to enable encrypted operations without decryption. Nevertheless, together with a robust security guarantee, homomorphic encryption brings extra communication and computation overhead. In this paper, we analyze the current bottlenecks of vertical federated learning under homomorphic encryption comprehensively and numerically. We propose a straggler-resilient and computation-efficient accelerating system that reduces the communication overhead in heterogeneous scenarios by 65.26% at most and reduces the computation overhead caused by homomorphic encryption by 40.66% at most. Our system can improve the robustness and efficiency of the current vertical federated learning framework without loss of security.

show abstract

“…In [6], a VFL design with the soft decision mode is considered and a RNN-based model is used to exploit the consistency of data samples in the time domain to improve the detection accuracy. The convergence performance of the standard VFL algorithm is analyzed in [7]. However, in the above works [5]- [7], the authors consider the case that all SUs participate in each round, where all local models and the central model are trained with the aid of data exchange between the server and the SUs.…”

Section: Introductionmentioning

confidence: 99%

“…The convergence performance of the standard VFL algorithm is analyzed in [7]. However, in the above works [5]- [7], the authors consider the case that all SUs participate in each round, where all local models and the central model are trained with the aid of data exchange between the server and the SUs. In this case, high communication latency takes place once if some wireless links experience deep channel fadings.…”

Section: Introductionmentioning

confidence: 99%

Low-Latency Cooperative Spectrum Sensing via Truncated Vertical Federated Learning

Zhang¹,

Zhu²,

Cui³

2022

Preprint

View full text Add to dashboard Cite

In recent years, the exponential increase in the demand of wireless data transmission rises the urgency for accurate spectrum sensing approaches to improve spectrum efficiency. The unreliability of conventional spectrum sensing methods by using measurements from a single secondary user (SU) has motivated researches on cooperative spectrum sensing (CSS). In this work, we propose a vertical federated learning (VFL) framework to exploit the distributed features across multiple SUs without compromising the data privacy. However, the repetitive training process in VFL faces the issue of high communication latency. To accelerate the training process, we propose a truncated vertical federated learning (T-VFL) algorithm, where the training latency is highly reduced by integrating the standard VFL algorithm with a channel-aware user scheduling policy. The convergence performance of T-VFL is provided via mathematical analysis and justified by simulation results. Moreover, to guarantee the convergence performance of the T-VFL algorithm, we conclude three design rules on the neural architectures used under the VFL framework, whose effectiveness is proved through simulations.

show abstract

FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features

Cited by 47 publications

References 24 publications

A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning

A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning

Accelerating Vertical Federated Learning

Low-Latency Cooperative Spectrum Sensing via Truncated Vertical Federated Learning

Contact Info

Product

Resources

About