Federated learning (FL) emerges as a popular distributed learning schema that learns a model from a set of participating users without requiring raw data to be shared. One major challenge of FL comes from heterogeneity in users, which may have distributionally different (or non-iid) data and varying computation resources. Just like in centralized learning, FL users also desire model robustness against malicious attackers at test time. Whereas adversarial training (AT) provides a sound solution for centralized learning, extending its usage for FL users has imposed significant challenges, as many users may have very limited training data as well as tight computational budgets, to afford the data-hungry and costly AT. In this paper, we study a novel learning setting that propagates adversarial robustness from highresource users that can afford AT, to those low-resource users that cannot afford it, during the FL process. We show that existing FL techniques cannot effectively propagate adversarial robustness among non-iid users, and propose a simple yet effective propagation approach that transfers robustness through carefully designed batch-normalization statistics. We demonstrate the rationality and effectiveness of our method through extensive experiments. Especially, the proposed method is shown to grant FL remarkable robustness even when only a small portion of users afford AT during learning. Codes will be published upon acceptance.Preprint. Under review.
Federated learning is a distributed learning framework that is communication efficient and provides protection over participating users' raw training data. One outstanding challenge of federate learning comes from the users' heterogeneity, and learning from such data may yield biased and unfair models for minority groups. While adversarial learning is commonly used in centralized learning for mitigating bias, there are significant barriers when extending it to the federated framework. In this work, we study these barriers and address them by proposing a novel approach Federated Adversarial DEbiasing (FADE). FADE does not require users' sensitive group information for debiasing and offers users the freedom to optout from the adversarial component when privacy or computational costs become a concern. We show that ideally, FADE can attain the same global optimality as the one by the centralized algorithm. We then analyze when its convergence may fail in practice and propose a simple yet effective method to address the problem. Finally, we demonstrate the effectiveness of the proposed framework through extensive empirical studies, including the problem settings of unsupervised domain adaptation and fair learning. Our codes and pretrained models are available at: https://github.com/illidanlab/FADE.
Background Early identification and accurate assessment of Mild Cognitive Impairment (MCI) is critical for clinical‐trial enrichment as well as the early intervention of the neurodegenerative disease. Continuous home‐based measurements of functions using simple embedded sensors and devices could provide an opportunity to improve the sensitivity and specificity in identifying MCI subjects in the community. However, a large number of assessment data points from each individual might increase the possibility of a chance finding. Careful and creative approaches are required to confirm robustness of the findings. Methods A total of 152 participants enrolled in the Intelligent Systems for Assessing Aging Changes Study (ISAAC) from the Oregon Center for Aging and Technology (ORCATECH) were used in the current project. In‐home monitored functional measurements including daily average walking speed, walking variability, computer usage and other functional measures were used. Each subject contributed 1372 days of data on average (SD = 662.3). In order to reflect the fact that various stages of MCI and NC are enrolled into trials in real‐world, longitudinal trajectories from each participant was sliced into small pieces, i.e., slicing windows of different sizes and used for cross‐validation. The cross‐validation was conducted on randomly permutated left‐out validation data to evaluate the performance. We also examined proportion of missingness during each slicing window and examined its impact on the ability in identifying MCI. We used various feature selection methods including F1‐score, mutual information, and chi2 to rank the effectiveness of variables. From the selected variables, we constructed different machine learning models, including logistic regression and support vector machines. Results Models using top‐4 features gave the best ability in differentiating MCI from NC, indicated by Receiver Operating Characteristics Area under Curve. Using slicing windows with a missing ratio higher than 30% was found to compromise the model performance. Longer subsequences have more information for modeling, but subsequences longer than a 2‐month window did not further help the performance. Conclusions This study proposed an approach that effectively detects MCI from in‐home monitoring measurement, which could benefit study enrichment in preclinical trails. The approaches presented could aid in improving scientific rigor of digital biomarker analyses.
In this paper, we focus on subspace-based learning problems, where data elements are linear subspaces instead of vectors. To handle this kind of data, Grassmann kernels were proposed to measure the space structure and used with classifiers, e.g., Support Vector Machines (SVMs). However, the existing discriminative algorithms mostly ignore the instability of subspaces, which would cause the classifiers to be misled by disturbed instances. Thus we propose considering all potential disturbances of subspaces in learning processes to obtain more robust classifiers. Firstly, we derive the dual optimization of linear classifiers with disturbances subject to a known distribution, resulting in a new kernel, Disturbance Grassmann (DG) kernel. Secondly, we research into two kinds of disturbance, relevant to the subspace matrix and singular values of bases, with which we extend the Projection kernel on Grassmann manifolds to two new kernels. Experiments on action data indicate that the proposed kernels perform better compared to state-of-the-art subspace-based methods, even in a worse environment.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.