Abstract:Biological systems, including human beings, have the innate ability to perform complex tasks in versatile and agile manner. Researchers in sensorimotor control have tried to understand and formally define this innate property. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally lead to the formulation of the motor primitives theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant ma… Show more
“…There has also been extensive research in incorporating compositional temporal structure for multi-frequency robot control: from constructing a hierarchical abstraction of control primitives, to combining them with reinforcement learning. Early work in this area includes Dynamic Movement Primitives (DMPs) [21], [22], [23], [24], [25], [26], which use attractor dynamics to produce stable units of control that are sequenced or blended together to perform downstream tasks with imitation learning. DMPs have also since been extended and used within hierarchical reinforcement learning (RL) using the options framework [27], [28], [29], [30], where they are formulated as pretrained low-level skills that are composed hierarchically by having a high-level policy choose between primitive actions or pretrained skills [31].…”
We investigate pneumatic non-prehensile manipulation (i.e., blowing) as a means of efficiently moving scattered objects into a target receptacle. Due to the chaotic nature of aerodynamic forces, a blowing controller must (i) continually adapt to unexpected changes from its actions, (ii) maintain finegrained control, since the slightest misstep can result in large unintended consequences (e.g., scatter objects already in a pile), and (iii) infer long-range plans (e.g., move the robot to strategic blowing locations). We tackle these challenges in the context of deep reinforcement learning, introducing a multi-frequency version of the spatial action maps framework. This allows for efficient learning of vision-based policies that effectively combine high-level planning and low-level closed-loop control for dynamic mobile manipulation. Experiments show that our system learns efficient behaviors for the task, demonstrating in particular that blowing achieves better downstream performance than pushing, and that our policies improve performance over baselines. Moreover, we show that our system naturally encourages emergent specialization between the different subpolicies spanning low-level fine-grained control and high-level planning. On a real mobile robot equipped with a miniature air blower, we show that our simulation-trained policies transfer well to a real environment and can generalize to novel objects.
“…There has also been extensive research in incorporating compositional temporal structure for multi-frequency robot control: from constructing a hierarchical abstraction of control primitives, to combining them with reinforcement learning. Early work in this area includes Dynamic Movement Primitives (DMPs) [21], [22], [23], [24], [25], [26], which use attractor dynamics to produce stable units of control that are sequenced or blended together to perform downstream tasks with imitation learning. DMPs have also since been extended and used within hierarchical reinforcement learning (RL) using the options framework [27], [28], [29], [30], where they are formulated as pretrained low-level skills that are composed hierarchically by having a high-level policy choose between primitive actions or pretrained skills [31].…”
We investigate pneumatic non-prehensile manipulation (i.e., blowing) as a means of efficiently moving scattered objects into a target receptacle. Due to the chaotic nature of aerodynamic forces, a blowing controller must (i) continually adapt to unexpected changes from its actions, (ii) maintain finegrained control, since the slightest misstep can result in large unintended consequences (e.g., scatter objects already in a pile), and (iii) infer long-range plans (e.g., move the robot to strategic blowing locations). We tackle these challenges in the context of deep reinforcement learning, introducing a multi-frequency version of the spatial action maps framework. This allows for efficient learning of vision-based policies that effectively combine high-level planning and low-level closed-loop control for dynamic mobile manipulation. Experiments show that our system learns efficient behaviors for the task, demonstrating in particular that blowing achieves better downstream performance than pushing, and that our policies improve performance over baselines. Moreover, we show that our system naturally encourages emergent specialization between the different subpolicies spanning low-level fine-grained control and high-level planning. On a real mobile robot equipped with a miniature air blower, we show that our simulation-trained policies transfer well to a real environment and can generalize to novel objects.
“…DMP framework provides an elegant way to encode any arbitrary spatial trajectory as a stable second-order nonlinear system, which is well suited and widely utilized controlling for robotic systems [21]. The standard DMP system consists of a point attractor formulated as a second-order ordinary differential equation (ODE) with a nonlinear forcing term.…”
Section: Dynamic Movement Primitivesmentioning
confidence: 99%
“…Here xg is an estimate of the final goal position. A fair assumption is that the goal is moving slow enough such that v ≥ ẋg the convergence properties holds for the system in (21). The estimate of the final goal position at time t is updated with a simple weighted average of the current goal position and the position estimated using the goal velocity at time t,…”
Section: Moving Target Dmp With Velocity Feedbackmentioning
confidence: 99%
“…The goal position is updated in realtime based on a simple estimate from equation (22). The DMP system is simulated with a velocity feedback from the moving goal based on the transformation system in (21). For all the scenarios, the real-time behaviour of DMP systems with both exponential and polynomial canonical systems described by ( 2) and ( 15) respectively are shown in figure 4.…”
This work is aimed at extending the standard dynamic movement primitives (DMP) framework to adapt to real-time changes in the task execution time while preserving its style characteristics. We propose an alternative polynomial canonical system and an adaptive law allowing a higher degree of control over the execution time. The extended framework has a potential application in robotic manipulation tasks that involve moving objects demanding real-time control over the task execution time. The existing methods require a computationally expensive forward simulation of DMP at every time step which makes it undesirable for integration in realtime control systems. To address this deficiency, the behaviour of the canonical system has been adapted according to the changes in the desired execution time of the task performed. An alternative polynomial canonical system is proposed to provide increased real-time control on the temporal scaling of DMP system compared to the standard exponential canonical system. The developed method was evaluated on scenarios of tracking a moving target where the desired tracking time is varied in real-time. The results presented show that the extended version of DMP provide better control over the temporal scaling during the execution of the task. We have evaluated our approach on a UR5 robotic manipulator for tracking a moving object.
“…Thus, our framework can handle the Cartesian term q ∈ R and the Riemannian term v ∈ S 2 respectively. In brief, comparing with the state-of-theart researches [28], our framework can learn the multispace skills in cartesian space and 2D sphere manifold. The demonstrated human arm endpoint poses including positions and orientations can be transferred to robots simultaneously.…”
Dynamic movement primitives (DMPs) as a robust and efficient framework has been studied widely for robot learning from demonstration. Classical DMPs framework mainly focuses on the movement learning in Cartesian or joint space, and can't properly represent end-effector orientation. In this paper, we present an extended DMPs framework (EDMPs) both in Cartesian space and 2-Dimensional (2D) sphere manifold for Quaternion-based orientation learning and generalization. Gaussian mixture model and Gaussian mixture regression (GMM-GMR) are adopted as the initialization phase of EDMPs to handle multi-demonstrations and obtain their mean and covariance. Additionally, some evaluation indicators including reachability and similarity are defined to characterize the learning and generalization abilities of EDMPs. Finally, a real-world experiment was conducted with human demonstrations, the endpoint poses of human arm were recorded and successfully transferred from human to the robot. The experimental results show that the absolute errors of the Cartesian and Riemannian space skills are less than 3.5 mm and 1.0°, respectively. The Pearson’s correlation coefficients of the Cartesian and Riemannian space skills are mostly greater than 0.9. The developed EDMPs exhibits superior reachability and similarity for the multi-space skills’ learning and generalization. This research proposes a fused framework with EDMPs and GMM-GMR which has sufficient capability to handle the multi-space skills in multi-demonstrations.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.