Multi-Tier Federated Learning for Vertically Partitioned Data

Abstract

We consider decentralized model training in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. To reduce communication overhead, the clients in each silo perform multiple local gradient steps before sharing updates with their hub. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions, the number of local updates, and the number of clients in each hub. We further validate our approach empirically via simulation-based experiments using a variety of datasets and both convex and non-convex objectives.

Task-Driven Transferred Vertical Federated Deep Learning for Multivariate Internet of Things Time-Series Analysis

“…There are some methods that applied Split Learning to a VFDL scenario based on an active/passive party architecture [12,13]. In addition to vertical federated learning scenarios, Split Learning can also be applied to horizontal federated learning scenarios [16,27] or hybrid architectures [28,29].…”

SoK: Secure Aggregation Based on Cryptographic Schemes for Federated Learning

“…Hence, each client is able to train a local model on these samples. Differently, in a vertically partitioned dataset [DP21], a client may hold part of the features of each training sample while the other parts might be held by other FL clients. In this FL type, the clients are not able to locally train a model without collecting the missing information of each sample from other clients.…”

Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning

“…A client uses the same subset of sample IDs 0 , for all iterations. During local training, when a client executes (5), it recalculates its own local intermediate information Φ Remark 2. We assume that every client has the labels for all of its samples.…”

“…This work is supported by the Rensselaer-IBM AI Research Collaboration (http://airc.rpi.edu), part of the IBM AI Horizons Network (http://ibm.biz/AIHorizons), and by the National Science Foundation under grants CNS 1553340 and CNS 1816307. A preliminary conference version of this work has been published in [5].…”