Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Yu, Xianjia; Queralta, Jorge Peña; Westerlund, Tomi

doi:10.48550/arxiv.2204.06949

Cited by 2 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: A Data Collection For Flmentioning

confidence: 99%

“…The details of the data gathering, including the usage of the photorealistic simulator Nvidia Isaac Sim, the environment settings, and data distributions, are detailed in [5], from where we reuse the base simulation and training datasets. The datasets from [5] include data from three distinct scenarios in NVIDIA Isaac Sim and from Jetbot robots deployed in three realworld rooms to train and validate the vision-based obstacle avoidance models. In this work, the datasets from the simulator are represented as S i, i∈{0,1,2} where i indicates the simulated environments, including a hospital, office, and warehouse.The three real-world datasets are denoted as R i, i∈{0,1,2} where i indicates office spaces, hallways, and laboratory environments, respectively.…”

Section: A Data Collection For Flmentioning

confidence: 99%

“…Both simulated and real-world data are evaluated separately in this part. Extending the previous works in [5], by including the AlextNet and ResNet18 in this study, we can compare how the performance will vary (b) Federated learning (Sim)…”

Section: A Centralized Training Vs Federated Learningmentioning

confidence: 99%

“…In this paper, we extend the previous work [5] on federated learning for vision-based obstacle avoidance in mobile robots towards analyzing different architectures and, most importantly, introducing an strategy for continuous learning from both simulated and real robots carrying out different missions requiring autonomous navigation. There are a number of works in the literature showing the potential of DL for vision-based obstacle avoidance [6], and works integrating other techniques such multi-view structure-from-motion for enhanced performance [7].…”

Section: Introductionmentioning

confidence: 99%

“…There are a number of works in the literature showing the potential of DL for vision-based obstacle avoidance [6], and works integrating other techniques such multi-view structure-from-motion for enhanced performance [7]. However, in addition to the use of DL in [5], the methods are also validated in the real world, a federated learning (FL) framework is introduced, and the effects of learning through heterogeneous environments are analyzed.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

Yu¹,

Queralta²,

Westerlund³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The role of deep learning (DL) in robotics has significantly deepened over the last decade. Intelligent robotic systems today are highly connected systems that rely on DL for a variety of perception, control and other tasks. At the same time, autonomous robots are being increasingly deployed as part of fleets, with collaboration among robots becoming a more relevant factor. From the perspective of collaborative learning, federated learning (FL) enables continuous training of models in a distributed, privacy-preserving way. This paper focuses on vision-based obstacle avoidance for mobile robot navigation. On this basis, we explore the potential of FL for distributed systems of mobile robots enabling continuous learning via the engagement of robots in both simulated and real-world scenarios. We extend previous works by studying the performance of different image classifiers for FL, compared to centralized, cloud-based learning with a priori aggregated data. We also introduce an approach to continuous learning from mobile robots with extended sensor suites able to provide automatically labelled data while they are completing other tasks. We show that higher accuracies can be achieved by training the models in both simulation and reality, enabling continuous updates to deployed models.

show abstract

Section: A Data Collection For Flmentioning

confidence: 99%

Section: A Data Collection For Flmentioning

confidence: 99%

Section: A Centralized Training Vs Federated Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

Yu¹,

Queralta²,

Westerlund³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Distributed Anomaly Detection in Smart Grids: A Federated Learning-Based Approach

et al. 2023

View full text Add to dashboard Cite

The smart grid integrates Information and Communication Technologies (ICT) into the traditional power grid to manage the generation, distribution, and consumption of electrical energy. Despite its many advantages, it faces significant challenges, such as detecting abnormal behaviours in the grid. Identifying anomalous behaviours helps to discover unusual user power consumption, faulty infrastructure, power outages, equipment failures, energy thefts, or cyberattacks. Machine learning (ML)-based techniques on smart meter data has shown remarkable results in anomaly detection. However, traditional ML-based anomaly detection requires smart meters to share local data with a central server, which raises concerns regarding data security and user privacy. Server-based model training faces additional challenges, such as the requirement of centralised computing power, reliable network communication, large bandwidth capacity, and latency issues, all of which affect the real-time anomaly detection performance. Motivated by these concerns, we propose a Federated Learning (FL)-based smart grid anomaly detection scheme where ML models are trained locally in smart meters without sharing data with a central server, thus ensuring user privacy. In the proposed approach, a global model is downloaded from the server to smart meters for ondevice training. After local training, local model parameters are sent to the server to improve the global model. We secure the model parameter updates from adversaries using the SSL/TLS protocol. Using standard datasets, we investigate the anomaly detection performance of federated learning and observe that FL models achieve anomaly detection performance comparable to centralised ML models while ensuring user privacy. Further, our study shows that the proposed FL-based models perform efficiently in terms of memory, CPU usage, bandwidth and power consumption at edge devices and are suitable for implementation in resourceconstrained environments, such as smart meters, for anomaly detection.

show abstract

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Cited by 2 publications

References 18 publications

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

Distributed Anomaly Detection in Smart Grids: A Federated Learning-Based Approach

Contact Info

Product

Resources

About