Supervised learning and reinforcement learning of feedback models for reactive behaviors: Tactile feedback testbed

Abstract

Robots need to be able to adapt to unexpected changes in the environment such that they can autonomously succeed in their tasks. However, hand-designing feedback models for adaptation is tedious, if at all possible, making data-driven methods a promising alternative. In this paper, we introduce a full framework for learning feedback models for reactive motion planning. Our pipeline starts by segmenting demonstrations of a complete task into motion primitives via a semi-automated segmentation algorithm. Then, given additional demonstrations of successful adaptation behaviors, we learn initial feedback models through learning-from-demonstrations. In the final phase, a sample-efficient reinforcement learning algorithm fine-tunes these feedback models for novel task settings through few real system interactions. We evaluate our approach on a real anthropomorphic robot in learning a tactile feedback task.

AL-ProMP: Force-relevant skills learning and generalization method for robotic polishing

“…To incorporate the movement phase dependency into the feedback model, Sutanto et al [33] proposed phase-modulated neural networks (PMNNs), which could learn phase-dependent feedback models. Building upon this, Sutanto et al [1] presented a full framework for learning feedback models for reactive motion planning and used a sample-efficient RL algorithm to fine-tune these feedback models for novel tasks through a limited number of interactions with the real system. It is worth noting that all these sensor feedback models are involved in the tuning of the skill model as one term of the DMPs.…”

“…Moreover, to address the challenges of gradient explosion and gradient disappearance, the gate recurrent unit (GRU) is used to learn time series information, as it offers computational efficiency compared to the long short-term memory (LSTM) [34]. Furthermore, the phases of actions are incorporated into the network construction to make the feedback model dependent on the evolution of phases [1], enabling improved scalability of the skill model in the time domain. Fig.…”

“…Learning from demonstration (LfD) is increasingly being used for robotic contact tasks, including tactile tasks [1][2] [3], assembly tasks [4] [5] [6], cutting [7] [8], writing [9] and polishing [10] [11]. Traditional kinematics-based skill learning often overlooks vital force and stiffness information, making it challenging to apply effectively to intricate tasks involving complex force interactions [12] [13].…”

“…Force signals measured by force sensors are the basis of this work. Mapping the contact force error to the adjustment amount of the skills model is one of the ways to address this problem [1], which can enhance the robustness of the contact task skills model. Initially, some handdesigned feedback models [14] [15] were used to map the sensor-measured error to the amount of movement primitive correction.…”

“…PMDRNN takes into account the interactive dynamics between the tool and the workpiece, as well as the effect of adjacent moments, and the effect of the phase function in the skill model DMP, which satisfies the learning of dynamical system features associated with the canonical system in DMPs. By combining the phase-modulated term [1] with the diagonal recurrent neural network (DRNN) [19] to improve the accuracy of the force sensor feedback learning model.…”