Humanoid robots are equipped with humanoid arms to make them more acceptable to the general public. Humanoid robots are a great challenge in robotics. The concept of digital twin technology complies with the guiding ideology of not only Industry 4.0, but also Made in China 2025. This paper proposes a scheme that combines deep reinforcement learning (DRL) with digital twin technology for controlling humanoid robot arms. For rapid and stable motion planning for humanoid robots, multitasking-oriented training using the twin synchro-control (TSC) scheme with DRL is proposed. For switching between tasks, the robot arm training must be quick and diverse. In this work, an approach for obtaining a priori knowledge as input to DRL is developed and verified using simulations. Two simple examples are developed in a simulation environment. We developed a data acquisition system to generate angle data efficiently and automatically. These data are used to improve the reward function of the deep deterministic policy gradient (DDPG) and quickly train the robot for a task. The approach is applied to a model of the humanoid robot BHR-6, a humanoid robot with multiple-motion mode and a sophisticated mechanical structure. Using the policies trained in the simulations, the humanoid robot can perform tasks that are not possible to train with existing methods. The training is fast and allows the robot to perform multiple tasks. Our approach utilizes human joint angle data collected by the data acquisition system to solve the problem of a sparse reward in DRL for two simple tasks. A comparison with simulation results for controllers trained using the vanilla DDPG show that the designed controller developed using the DDPG with the TSC scheme have great advantages in terms of learning stability and convergence speed.
The spring-loaded inverted pendulum model is similar to human walking in terms of the center of mass (CoM) trajectory and the ground reaction force. It is thus widely used in humanoid robot motion planning. A method that uses a velocity feedback controller to adjust the landing point of a robot leg is inaccurate in the presence of disturbances and a nonlinear optimization method with multiple variables is complicated and thus unsuitable for real-time control. In this paper, to achieve real-time optimization, a CoM-velocity feedback controller is used to calculate the virtual landing point. We construct a touchdown return map based on a virtual landing point and use nonlinear least squares to optimize spring stiffness. For robot whole-body control, hierarchical quadratic programming optimization is used to achieve strict task priority. The dynamic equation is given the highest priority and inverse dynamics are directly used to solve it, reducing the number of optimizations. Simulation and experimental results show that a force-controlled biped robot with the proposed method can stably walk on unknown uneven ground with a maximum obstacle height of 5 cm. The robot can recover from a 5 Nm disturbance during walking without falling.
An optimization framework for upward jumping motion based on quadratic programming (QP) is proposed in this paper, which can simultaneously consider constraints such as the zero moment point (ZMP), limitation of angular accelerations, and anti-slippage. Our approach comprises two parts: the trajectory generation and real-time control. In the trajectory generation for the launch phase, we discretize the continuous trajectories and assume that the accelerations between the two sampling intervals are constant and transcribe the problem into a nonlinear optimization problem. In the real-time control of the stance phase, the over-constrained control objectives such as the tracking of the center of moment (CoM), angle, and angular momentum, and constraints such as the anti-slippage, ZMP, and limitation of joint acceleration are unified within a framework based on QP optimization. Input angles of the actuated joints are thus obtained through a simple iteration. The simulation result reveals that a successful upward jump to a height of 16.4 cm was achieved, which confirms that the controller fully satisfies all constraints and achieves the control objectives.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.