Efficient and Fair Data Valuation for Horizontal Federated Learning

Wei, Shuyue; Tong, Yongxin; Zhou, Zimu; Song, Tianshu

doi:10.1007/978-3-030-63076-8_10

Cited by 37 publications

(34 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The diference is that MR calculates SVs during every round of FL model training, while OR calculates SV only once after all the training rounds are complete. In [26], the authors proposed the Truncated MR (TMR) approach, which extends MR by eliminating the entire rounds with decay factor λ values lower than a pre-deined threshold. These three schemes signiicantly improve the eiciency of estimating SVs.…”

Section: Fl Participant Contribution Evaluation Via Gradient Shapley ...mentioning

confidence: 99%

“…We analyze the MR approach [26], which combines the canonical SV calculation and gradient-based model reconstruction. We adopt the non-i.i.d FL data silos setting as described in the experiment section with 10 participants and MNIST dataset [11].…”

Section: Preliminariesmentioning

confidence: 99%

“…1 5.1.1 Datasets. We adopt the FL data silos settings in [26]. The dataset used in the experiments is based on the MNIST dataset [11].…”

Section: Experiments Setingsmentioning

confidence: 99%

“…The diferences are 1) the subsets performances used for estimating Shapley diferences are evaluated on sub-model reconstructed by participants model parameters, and 2) SVs are estimated independently each round and aggregated later. (6) TMR: Under this method [26], SVs are calculated independently in each round with participants' gradient updates, with a decay parameter λ which serves as 1) weights to amplify SVs from earlier rounds, and 2) as a truncation factor to eliminate unnecessary sub-model re-constructions. The convergence criterion for all approaches is set as:…”

Section: Comparisonmentioning

confidence: 99%

See 3 more Smart Citations

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Liu

Chen

et al. 2022

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Federated Learning (FL) bridges the gap between collaborative machine learning and preserving data privacy. To sustain the long-term operation of an FL ecosystem, it is important to attract high-quality data owners with appropriate incentive schemes. As an important building block of such incentive schemes, it is essential to fairly evaluate participants’ contribution to the performance of the final FL model without exposing their private data. Shapley Value (SV)-based techniques have been widely adopted to provide a fair evaluation of FL participant contributions. However, existing approaches incur significant computation costs, making them difficult to apply in practice. In this paper, we propose the Guided Truncation Gradient Shapley (GTG-Shapley) approach to address this challenge. It reconstructs FL models from gradient updates for SV calculation instead of repeatedly training with different combinations of FL participants. In addition, we design a guided Monte Carlo sampling approach combined with within-round and between-round truncation to further reduce the number of model reconstructions and evaluations required, through extensive experiments under diverse realistic data distribution settings. The results demonstrate that GTG-Shapley can closely approximate actual Shapley values, while significantly increasing computational efficiency compared to the state of the art, especially under non-i.i.d. settings.

show abstract

Section: Fl Participant Contribution Evaluation Via Gradient Shapley ...mentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

“…1 5.1.1 Datasets. We adopt the FL data silos settings in [26]. The dataset used in the experiments is based on the MNIST dataset [11].…”

Section: Experiments Setingsmentioning

confidence: 99%

Section: Comparisonmentioning

confidence: 99%

See 2 more Smart Citations

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Liu

Chen

et al. 2022

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

show abstract

“…It is essential to understand how to evaluate the contributions of data owners based on federated learning. Wei et al, 2020;Wang et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Improving Fairness for Data Valuation in Horizontal Federated Learning

Fan,

Fang,

Zhou

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances.

show abstract

Effective knowledge representation and utilization for sustainable collaborative learning across heterogeneous systems

Hoang

2024

AI Magazine

View full text Add to dashboard Cite

The increasingly decentralized and private nature of data in our digital society has motivated the development of collaborative intelligent systems that enable knowledge aggregation among data owners. However, collaborative learning has only been investigated in simple settings. For example, clients are often assumed to train solution models de novo, disregarding all prior expertise. The learned model is typically represented in task‐specific forms that are not generalizable to unseen, emerging scenarios. Finally, a universal model representation is enforced among collaborators, ignoring their local compute constraints or input representations. These limitations hampers the practicality of prior collaborative systems in learning scenarios with limited task data that demand constant knowledge adaptation and transfer across information silos, tasks, and learning models, as well as the utilization of prior solution expertise. Furthermore, prior collaborative learning frameworks are not sustainable on a macro scale where participants desire fairness allocation of benefits (e.g., access to the combined model) based on their costs of participation (e.g., overhead of model sharing and training synchronization, risk of information breaches, etc.). This necessitates a new perspective of collaborative learning where the server not only aggregates but also conducts valuation of the participant's contribution, and distribute aggregated information to individuals in commensurate to their contribution. To substantiate the above vision, we propose a new research agenda on developing effective and sustainable collaborative learning frameworks across heterogeneous systems, featuring three novel computational capabilities on knowledge organization: model expression, comprehension, and valuation.

show abstract

Efficient and Fair Data Valuation for Horizontal Federated Learning

Cited by 37 publications

References 3 publications

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Improving Fairness for Data Valuation in Horizontal Federated Learning

Effective knowledge representation and utilization for sustainable collaborative learning across heterogeneous systems

Contact Info

Product

Resources

About