DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning

Yu, Javier; Vincent, Joseph A.; Schwager, Mac

doi:10.1109/lra.2022.3142402

Cited by 19 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is because, depending on your computing power, the system may be able to process more samples at one time. For training with large amounts of data such as MNIST, researchers prefer to use batch size to avoid computer overload and speed up training [37]. It is chosen for this study to see if the batch size seems to affect your training.…”

Section: Figure 3: Impact Of Dropout On Standard Neural Networkmentioning

confidence: 99%

An Optimization Approach for Convolutional Neural Network Using Non-Dominated Sorted Genetic Algorithm-II

Zafar¹,

Aamir²,

Nawi³

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

In computer vision, convolutional neural networks have a wide range of uses. Images represent most of today's data, so it's important to know how to handle these large amounts of data efficiently. Convolutional neural networks have been shown to solve image processing problems effectively. However, when designing the network structure for a particular problem, you need to adjust the hyperparameters for higher accuracy. This technique is time consuming and requires a lot of work and domain knowledge. Designing a convolutional neural network architecture is a classic NP-hard optimization challenge. On the other hand, different datasets require different combinations of models or hyperparameters, which can be time consuming and inconvenient. Various approaches have been proposed to overcome this problem, such as grid search limited to low-dimensional space and queuing by random selection. To address this issue, we propose an evolutionary algorithm-based approach that dynamically enhances the structure of Convolution Neural Networks (CNNs) using optimized hyperparameters. This study proposes a method using Non-dominated sorted genetic algorithms (NSGA) to improve the hyperparameters of the CNN model. In addition, different types and parameter ranges of existing genetic algorithms are used. A comparative study was conducted with various state-of-the-art methodologies and algorithms. Experiments have shown that our proposed approach is superior to previous 5642 CMC, 2023, vol.74, no.3 methods in terms of classification accuracy, and the results are published in modern computing literature.

show abstract

Section: Figure 3: Impact Of Dropout On Standard Neural Networkmentioning

confidence: 99%

An Optimization Approach for Convolutional Neural Network Using Non-Dominated Sorted Genetic Algorithm-II

Zafar¹,

Aamir²,

Nawi³

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…supervised learning. See [25] for an ADMM-based distributed optimization approach to solving this problem.…”

Section: E Multi-robot Learningmentioning

confidence: 99%

Distributed Optimization Methods for Multi-Robot Systems: Part I - A Tutorial

Shorinwa¹,

Halsted²,

Yu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Distributed optimization provides a framework for deriving distributed algorithms for a variety of multi-robot problems. This tutorial constitutes the first part of a twopart series on distributed optimization applied to multi-robot problems, which seeks to advance the application of distributed optimization in robotics. In this tutorial, we demonstrate that many canonical multi-robot problems can be cast within the distributed optimization framework, such as multi-robot simultaneous localization and planning (SLAM), multi-robot target tracking, and multi-robot task assignment problems. We identify three broad categories of distributed optimization algorithms: distributed first-order methods, distributed sequential convex programming, and the alternating direction method of multipliers (ADMM). We describe the basic structure of each category and provide representative algorithms within each category. We then work through a simulation case study of multiple drones collaboratively tracking a ground vehicle. We compare solutions to this problem using a number of different distributed optimization algorithms. In addition, we implement a distributed optimization algorithm in hardware on a network of Rasberry Pis communicating with XBee modules to illustrate robustness to the challenges of real-world communication networks.

show abstract

“…The authors decompose an original state into two sub-states and protect data privacy when interacting and communicating with multi-robot by adding perturbations, which can effectively resist external attacks. The authors of the literature [52] proposed a distributed multi-robot training framework based on the augmented Lagrangian function, which finally reaches a consensus on the local model weights. The method can effectively solve the single point of failure problem and is robust to mid-robot dropouts.…”

Section: Multi-robot Distributed Learning Privacy Protectionmentioning

confidence: 99%

Multi-Robot Privacy-Preserving Algorithms Based on Federated Learning: A Review

Peng,

Guo,

Bao

et al. 2023

CMC

View full text Add to dashboard Cite

The robotics industry has seen rapid development in recent years due to the Corona Virus Disease 2019. With the development of sensors and smart devices, factories and enterprises have accumulated a large amount of data in their daily production, which creates extremely favorable conditions for robots to perform machine learning. However, in recent years, people's awareness of data privacy has been increasing, leading to the inability to circulate data between different enterprises, resulting in the emergence of data silos. The emergence of federated learning provides a feasible solution to this problem, and the combination of federated learning and multi-robot systems can break down data silos and improve the overall performance of robots. However, as scholars have studied more deeply, they found that federated learning has very limited privacy protection. Therefore, how to protect data privacy from infringement remains an important issue. In this paper, we first give a brief introduction to the current development of multi-robot and federated learning; second, we review three aspects of privacy protection methods commonly used, privacy protection methods for multi-robot, and Other Problems Faced by Multi-robot Systems, focusing on method comparisons and challenges; and finally draw conclusions and predict possible future research directions.

show abstract

DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning

Cited by 19 publications

References 32 publications

An Optimization Approach for Convolutional Neural Network Using Non-Dominated Sorted Genetic Algorithm-II

An Optimization Approach for Convolutional Neural Network Using Non-Dominated Sorted Genetic Algorithm-II

Distributed Optimization Methods for Multi-Robot Systems: Part I - A Tutorial

Multi-Robot Privacy-Preserving Algorithms Based on Federated Learning: A Review

Contact Info

Product

Resources

About