Data-efficient Learning of Morphology and Controller for a Microrobot

Abstract

Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design. For most robots, this process is further complicated by the need, when validating the capabilities of the hardware to solve the desired task, to already have an appropriate controller, which is in turn designed and tuned for the specific hardware. In this paper, we propose a novel approach, HPC-BBO, to efficiently and automatically design hardware configurations, and evaluate them by also automatically tuning the corresponding controller. HPC-BBO is based on a hierarchical Bayesian optimization process which iteratively optimizes morphology configurations (based on the performance of the previous designs during the controller learning process) and subsequently learns the corresponding controllers (exploiting the knowledge collected from optimizing for previous morphologies). Moreover, HPC-BBO can select a "batch" of multiple morphology designs at once, thus parallelizing hardware validation and reducing the number of time-consuming production cycles. We validate HPC-BBO on the design of the morphology and controller for a simulated 6-legged microrobot. Experimental results show that HPC-BBO outperforms multiple competitive baselines, and yields a 360% reduction in production cycles over standard Bayesian optimization, thus reducing the hypothetical manufacturing time of our microrobot from 21 to 4 months.

On the challenges of jointly optimising robot morphology and control using a hierarchical optimisation scheme

“…As in the work of Cheney et al [3], the two algorithms studied in this article become trapped in a local optima w.r.t morphology. The hierarchical scheme used does not overcome this issue: this is in contrast to the findings of [10] although here this is almost certainly due to the vastly more complex morphological space used. MEL-LHS generates more diversity, but still results in sub-optimal body plans.…”

“…A potential drawback of the hierarchical approach is that decoupling the morphology and controller optimisers can prevent information sharing between them. In [10] Bayesian Optimisation approach is used for both morphology and control loops to design hexapod micro-robots, selected for its efficiency as all optimisation takes place in reality. Here, the controller optimisation process exploits knowledge collected from optimising previous morphologies, providing an information-sharing method that improves data-efficiency by removing the need to start from scratch.…”

“…More strongly, they suggest that simultaneously changing morphology and control parameters can even be counter-productive, as a controller is specialised for some particular morphology [3]. Liao et al [10] propose that using a hierarchical approach is preferable, in which given a batch of morphologies, each robot independently learns a controller, and then the performance of these learned prototypes influences selection of the next generation of morphologies.…”

“…The learning algorithm can itself be an evolutionary algorithm but alternative algorithms have also been used, e.g. Reinforcement Learning (RL) in [6], Bayesian Optimisation (BO) in [10]. The separation of the two loops brings an additional advantage compared to simultaneous optimisation when considering the co-design of robots that also need to be fabricated: while it is extremely costly (in terms of time and material) to generate a physical body, learning trials are relatively computationally cheap, hence it makes sense to be able to allocate more budget to learning controllers than bodies [10].…”

Learning directed locomotion in modular robots with evolvable morphologies

“…[13,12] present efficient gait learning on a physical bipedal robot within limited trials by Bayesian optimization. In recent studies [35,48], Bayesian optimization achieves data-efficient learning of locomotion on a 6-legged microrobot. Other successful approaches that have been extensively investigated for robot locomotion are based on Central Pattern Generators (CPG) [27].…”