Toward Understanding the Influence of Individual Clients in Federated Learning

Abstract

Federated learning allows mobile clients to jointly train a global model without sending their private data to a central server. Extensive works have studied the performance guarantee of the global model, however, it is still unclear how each individual client influences the collaborative training process. In this work, we defined a new notion, called {\em Fed-Influence}, to quantify this influence over the model parameters, and proposed an effective and efficient algorithm to estimate this metric. In particular, our design satisfies several desirable properties: (1) it requires neither retraining nor retracing, adding only linear computational overhead to clients and the server; (2) it strictly maintains the tenets of federated learning, without revealing any client's local private data; and (3) it works well on both convex and non-convex loss functions, and does not require the final model to be optimal. Empirical results on a synthetic dataset and the FEMNIST dataset demonstrate that our estimation method can approximate Fed-Influence with small bias. Further, we show an application of Fed-Influence in model debugging.

Empirical Measurement of Client Contribution for Federated Learning With Data Size Diversification

“…We use the permutation-based RoundSV, utilizing Monte-Carlo sampling for SV approximation. • Fed-Influence in Accuracy(FIA) [25] is a type of Fed-Influence measurement metric that simply measures the influence by investigating the effect of removing a client only. The actual FIA value can be obtained from the results of the leave-one-out test.…”

“…Local gradients, local weights, and local data sizes are the only possible information that the server can use as a tool for client contribution evaluation. [2], [20] In particular, LOO [14], [25] and Shapley Value [8] are applicable valuation methods in distributed systems that use local weights or gradients as a tool for client contribution evaluation. While these game-theoretic methods are timeconsuming, a simple approximation for LOO [25] and Shapley Value [21], [26], [27], [40] makes the client contribution calculation feasible with a theoretical base.…”

“…In addition, the combinatorial impact distracts the measurement of ground-truth values owing to the unclear linearity with data heterogeneity. Alternatively, the client removal test [8], [25], [35] is commonly used for the precision testing of client contribution measurements. As shown in Fig.…”

“…In contrast, the evaluation complexity of other baseline methods is highly dependent on the number of clients participating in FL. RoundSV costs O(N logN ) [8], FIA costs O(N 2 ) [25], and RRAFL costs O(N ).…”

“…We empirically analyze the robustness to diverse heterogeneous data situations and the practicality of data size selection. Leave-one-out [14], [25] Measure the marginal performance difference of a specific client's participation.…”