Modular Robot Design Synthesis with Deep Reinforcement Learning

Whitman, Julian; Bhirangi, Raunaq; Travers, Matthew; Choset, Howie

doi:10.1609/aaai.v34i06.6611

Cited by 27 publications

(23 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One such field is robotics [2,27]. For many of the works which adapt morphology in robotics, designs are first established in simulation and then later fabricated and tested in a realworld environment [8,21,40,47]. Learning physiology in conjunction with policy has also been used in the fields of evolutionary algorithms [5], deep learning [16], and deep reinforcement learning [24,35,48] which adapt simulated agents to succeed in entirely simulated tasks.…”

Section: Related Workmentioning

confidence: 99%

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

Pratt¹,

Weihs²,

Farhadi³

2022

Preprint

View full text Add to dashboard Cite

The last few years have witnessed substantial progress in the field of embodied AI where artificial agents, mirroring biological counterparts, are now able to learn from interaction to accomplish complex tasks. Despite this success, biological organisms still hold one large advantage over these simulated agents: adaptation. While both living and simulated agents make decisions to achieve goals (strategy), biological organisms have evolved to understand their environment (sensing) and respond to it (physiology). The net gain of these factors depends on the environment, and organisms have adapted accordingly. For example, in a low vision aquatic environment [31] some fish have evolved specific neurons which offer a predictable, but incredibly rapid, strategy to escape from predators [7]. Mammals have lost these reactive systems, but they have a much larger fields of view and brain circuitry capable of understanding many future possibilities [26]. While traditional embodied agents manipulate an environment to best achieve a goal, we argue for an introspective agent, which considers its own abilities in the context of its environment. We show that different environments yield vastly different optimal designs, and increasing long-term planning is often far less beneficial than other improvements, such as increased physical ability. We present these findings to broaden the definition of improvement in embodied AI passed increasingly complex models. Just as in nature, we hope to reframe strategy as one tool, among many, to succeed in an environment. Code is available at: https://github.com/ sarahpratt/introspective.

show abstract

Section: Related Workmentioning

confidence: 99%

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

Pratt¹,

Weihs²,

Farhadi³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23] explored different techniques for finding robot design such that the task-reaching a set of points in 3D-is feasible. While in [3,4,5,6,7,9,10,8] the authors analyzed kinematic requirements, such as dexterity [4,6], manipulability [10] and maximization of reachable space [5], in [11,12,23,13,15,16,17,18,20,19,21,22,14,20], the authors also incorporated torque specifications in their formulation.…”

Section: Related Workmentioning

confidence: 99%

“…In [18,16,17,19], the authors included static torque requirements in their approach. The total torque applied to the robot's actuator is composed of static and dynamic components.…”

Section: Related Workmentioning

confidence: 99%

Synthesizing Modular Manipulators For Tasks With Time, Obstacle, And Torque Constraints

Campos,

Kress-Gazit

2021

Preprint

View full text Add to dashboard Cite

Modular robots can be tailored to achieve specific tasks and rearranged to achieve previously infeasible ones. The challenge is choosing an appropriate design from a large search space. In this work, we describe a framework that automatically synthesizes the design and controls for a serial chain modular manipulator given a task description. The task includes points to be reached in the 3D space, time constraints, a load to be sustained at the end-effector, and obstacles to be avoided in the environment. These specifications are encoded as a constrained optimization in the robot's kinematics and dynamics and, if a solution is found, the formulation returns the specific design and controls to perform the task. Finally, we demonstrate our approach on a complex specification in which the robot navigates a constrained environment while holding an object.

show abstract

“…There also has been a large body of research on modular robots [24]- [36] where a set of predefined reusable modules is designed to compose versatile robotic systems to solve a wide variety of tasks on the fly. [29], [33] formulate the design search as a Markov Decision Process. [29] learn to perform composing actions such as "link" and "unlink" through deep reinforcement learning.…”

Section: Related Workmentioning

confidence: 99%

“…[29] learn to perform composing actions such as "link" and "unlink" through deep reinforcement learning. [33] train an action value network to predict the expected return of different assemblies to plan which module to add next. [35] develop a platform that enables a human-in-the-loop iterative design process for customized assembly.…”

Section: Related Workmentioning

confidence: 99%

Neural fidelity warping for efficient robot morphology design

Hu¹,

Yang²,

Mori³

2020

Preprint

View full text Add to dashboard Cite

We consider the problem of optimizing a robot morphology to achieve the best performance for a target task, under computational resource limitations. The evaluation process for each morphological design involves learning a controller for the design, which can consume substantial time and computational resources. To address the challenge of expensive robot morphology evaluation, we present a continuous multi-fidelity Bayesian Optimization framework that efficiently utilizes computational resources via low-fidelity evaluations. We identify the problem of non-stationarity over fidelity space. Our proposed fidelity warping mechanism can learn representations of learning epochs and tasks to model nonstationary covariances between continuous fidelity evaluations which prove challenging for off-the-shelf stationary kernels. Various experiments demonstrate that our method can utilize the low-fidelity evaluations to efficiently search for the optimal robot morphology, outperforming state-of-the-art methods.

show abstract

Modular Robot Design Synthesis with Deep Reinforcement Learning

Cited by 27 publications

References 12 publications

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

Synthesizing Modular Manipulators For Tasks With Time, Obstacle, And Torque Constraints

Neural fidelity warping for efficient robot morphology design

Contact Info

Product

Resources

About