Inspecting the Running Process of Horizontal Federated Learning via Visual Analytics

Abstract

As a decentralized training approach, horizontal federated learning (HFL) enables distributed clients to collaboratively learn a machine learning model while keeping personal/private information on local devices. Despite the enhanced performance and efficiency of HFL over local training, clues for inspecting the behaviors of the participating clients and the federated model are usually lacking due to the privacy-preserving nature of HFL. Consequently, the users can only conduct a shallow-level analysis of potential abnormal behaviors and have limited means to assess the contributions of individual clients and implement the necessary intervention. Visualization techniques have been introduced to facilitate the HFL process inspection, usually by providing model metrics and evaluation results as a dashboard representation. Although the existing visualization methods allow a simple examination of the HFL model performance, they cannot support the intensive exploration of the HFL process. In this study, strictly following the HFL privacy-preserving protocol, we design an exploratory visual analytics system for the HFL process termed HFLens, which supports comparative visual interpretation at the overview, communication round, and client instance levels. Specifically, the proposed system facilitates the investigation of the overall process involving all clients, the correlation analysis of clients' information in one or different communication round(s), the identification of potential anomalies, and the contribution assessment of each HFL client. Two case studies confirm the efficacy of our system. Experts' feedback suggests that our approach indeed helps in understanding and diagnosing the HFL process better.

Explainable Federated Learning: A Lifecycle Dashboard for Industrial Settings

“…FATEBoard shows basic information like running time, log outputs, and current status for different components while also giving an overview of the progress of jobs [13]. Unfortunately, no detailed information regarding client anomalies is provided [14]. Another alternative is the open-source framework NVIDIA FLARE, which adapts existing workflows for federation and provides visualizations during FL experiments using TensorBoard [15].…”

“…Using this system, users could see how much each client contributed and how the global model improved over time. One of the most comprehensive studies for visualizing FL was conducted by Liet al [14], which proposed a system for inspecting the running process in horizontal FL. In addition to basic visualization of metrics, they enable further client anomaly detection, pairwise comparison of clients, and contribution analysis.…”

“…Li et al [14] have studied what requirements a visualization system for horizontal FL needs to fulfill to meet the demands and expectations of experts. Expanding on this analysis, we also gathered feedback from industrial stakeholders to derive requirements and expected features for our representation approach.…”

“…System View: Service Engineers require an Client behavior: To better understand federated learning processes, it is important to know the status of clients and if they participated in communication rounds [14]. Especially during training, it is essential to see for each communication round which clients participated, how much they contributed, and how they fared compared to their peers.…”

“…The main components for the training lifecycle stage are shown in Figures 2c and 2d. Similar to the proposed representation approach by Li et al [14], this boxplot graph visualizes the model performance of clients in a cohort during training. The y-axis displays the elapsed training time in communication rounds, while the x-axis is used to plot boxplots of different metrics.…”