Distributed Particle Swarm Optimization using Optimal Computing Budget Allocation for multi-robot learning

Abstract

Abstract-Particle Swarm Optimization (PSO) is a populationbased metaheuristic that can be applied to optimize controllers for multiple robots using only local information. In order to cope with noise in the robotic performance evaluations, different reevaluation strategies were proposed in the past. In this article, we apply a statistical technique called Optimal Computing Budget Allocation to improve the performance of distributed PSO in the presence of noise. In particular, we compare a distributed PSO OCBA algorithm suitable for resource-constrained mobile robots with a centralized version that uses global information for the allocation. We show that the distributed PSO OCBA outperforms a previous distributed noise-resistant PSO variant, and that the performance of the distributed PSO OCBA approaches that of the centralized one as the communication radius is increased. We also explore different parametrizations of the PSO OCBA algorithm, and show that the choice of parameter values differs from previous guidelines proposed for stand-alone OCBA.

A Noise-Resistant Mixed-Discrete Particle Swarm Optimization Algorithm for the Automatic Design of Robotic Controllers

“…MDPSO-OCBA results not only in a better performing solution, but also in a significantly decreased the gap between recorded and re-evaluated cost, corresponding to a better assessment of the learning progress within PSO.Very similar results were obtained using the other scenarios and are thus not reproduced here. It is also worth noting that the results are coherent with the ones reported in [21] for continuous PSO.…”

“…We therefore have enhanced the original, MDPSO algorithm introduced in [12] with an OCBA scheme, in order to better handle the noise inherent to robotic systems as shown in Algorithm 1. Akin to OCBA C of [21], for every PSO iteration, after initial n 0 evaluations of each particle's solution, OCBA determines, based on means and standard deviations, how much of the total available iteration budget B i each candidate solution should be allocated in order to enable an optimal performance comparison between them. Note that the difference between OCBA C of [21] and this algorithm is in the way particle positions are updated, as shown in Algorithm 2.…”

“…However, as this mechanism does not always result in better optimization performance [12], we also consider a version of MDPSO without explicit diversity preservation in this work. This "MDPSO without diversity preservation" corresponds to a classical, OCBAenhanced noise-resistant PSO as in [21], with the additional rounding to obtain discrete variables as in [12].…”

Lightweight physics-based models for the control of fluid-mediated self-assembly of robotic modules

“…This motivates the choice of PSO for our particular optimization problem where stochasticity is an inherent feature of the underlying system. Multiple computationally efficient recipes for increasing the robustness of the PSO algorithm to noisy evaluations have been proposed in the literature [18,19]. In this work, we have opted for a simple solution involving a re-evaluation and aggregation of the personal best performance of each particle as suggested in [17].…”