An integrated vision-based system for efficient robot arm teleoperation

Wu, Xingkun; Yang, Canjun; Zhu, Yuanchao; Wu, Weitao; Wei, Qianxiao

doi:10.1108/ir-06-2020-0129

Cited by 8 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite depth cameras such as Kinect [43] being viable for motion capture, they easily experience infrared interference, which makes them vulnerable to failure in outdoor environments. In recent years, increasing interest in deep learning has driven the development of monocular image-based human pose estimation [44]- [46], which can be used when the required precision is not high. Multiple inertial sensors can also be employed to measure human motions in real time [47], using inverse kinematics to calculate human joint angles [48].…”

Section: A Perception Of Robots With Regard To Humansmentioning

confidence: 99%

A Review of Human–Machine Cooperation in the Robotics Domain

Yang

Zhu

Chen

2022

IEEE Trans. Human-Mach. Syst.

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI) technology has greatly expanded human capabilities through perception, understanding, action, and learning. The future of AI depends on cooperation between humans and AI. In addition to a fully automated or manually controlled machine, a machine can work in tandem with a human with different levels of assistance and automation. Machines and humans cooperate in different ways. Three strategies for cooperation are described in this article, as well as the nesting relationships among different control methods and cooperation strategies. Based on human thinking and behavior, a hierarchical human-machine cooperation (HMC) framework is improved and extended to design safe, efficient, and attractive systems. We review the common methods of perception, decision-making, and execution in the HMC framework. Future applications and trends of HMC are also discussed.

show abstract

Section: A Perception Of Robots With Regard To Humansmentioning

confidence: 99%

A Review of Human–Machine Cooperation in the Robotics Domain

Yang

Zhu

Chen

2022

IEEE Trans. Human-Mach. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Liu et al [8] designed a hapticbased teleoperation system for a large-sized space exploration slave robot, which is composed of 3R and DELTA mechanisms. Wu et al [9] proposed a vision-based human-robot teleoperation system based on monocular human pose estimation convolutional neural networks. In addition, virtual reality, mixed reality, and augmented reality technologies are applied to heterogeneous teleoperation systems to improve the intuitiveness and flexibility of the teleoperation systems [10]- [13].…”

Section: Introductionmentioning

confidence: 99%

Master-Slave Arm Heterogeneous Mapping With Link Pose Constraint in Teleoperation System

Zhang

Zhai

et al. 2022

IEEE Access

View full text Add to dashboard Cite

In order to solve the problem in master-slave heterogeneous teleoperation systems, which is that the inverse solution of robot cannot be achieved when the mapped wrist joint exceeds the reachable space of robot wrist joint, a master-slave heterogeneous mapping method based on link pose constraint is proposed. The link pose constraint means that the position and orientation of robot hand link are the same or similar to that of exoskeleton hand link. In the proposed method, the link pose constraint is used in an exoskeleton-based heterogeneous teleoperation system to realize consistent mapping between the exoskeleton and robot hand link inside wrist joint reachable space, and to realize effective inverse solution and consistent mapping of the end effector (EE) outside the reachable space. A simulation was performed by using the proposed method. The simulation results show that the proposed method can effectively realize the mapping between the exoskeleton and robot in the reachable space of robot EE. The teleoperation experimental setup was constructed by using the developed exoskeleton and an industrial robot, and the verification experiments of the proposed mapping method were performed. The experimental results indicate that the position error of the proposed heterogeneous mapping method is 0.48 mm. The experimental results demonstrate that the proposed mapping method can effectively realize master-slave heterogeneous mapping and realize continuous motion control of the teleoperation system, which satisfies the application requirements of human-robot integration, robot skill enhancement, and robot skill learning.

show abstract

“…For different robotic arms, some scholars first divided them into rod models corresponding to the rods of the human arm and then used the same direction of the corresponding rods as a similarity index to realize the mapping of the two arms (Liang et al , 2016; Gong et al , 2017). In Shahverdi and Masouleh (2017) and Wu et al (2020), the researchers built similarity metrics using geometric relationships, enabling motion mapping from human arms to humanoid robotic arms. Although the mapping method in the joint space can ensure the configuration similarity of the two arms, it cannot guarantee the accuracy of the end-effector.…”

Section: Introductionmentioning

confidence: 99%

Human-robot kinematics mapping method based on dynamic equivalent points

Zhao

Wang

Xie

et al. 2022

View full text Add to dashboard Cite

Purpose This paper aims to present a new kinematics mapping method based on dynamic equivalent points. In teleoperation, this method enables a robotic (follower) arm to mimic human (leader) arm postures and avoid obstacles in a human-like manner. Design/methodology/approach The information of the human arm is extracted based on the characteristics of human arm motion, and the concept of equivalent points is introduced. Then, an equivalent point is determined to transform the robotic arm with a nonhuman-like kinematic structure into an anthropomorphic robotic arm. Based on this equivalent point, a mapping method is developed to ensure that the two arms are similar. Finally, the similarity between the human elbow angle and robot elbow angle is further improved by using this method and an augmented Jacobian matrix with a compensation coefficient. Findings Numerical simulations and physical prototype experiments are conducted to verify the effectiveness and feasibility of the proposed method. In environments with obstacles, this method can adjust the position of the equivalent point in real time to avoid obstacles. In environments without obstacles, the similarity between the human elbow angle and robot elbow angle is further improved at the expense of the end-effector accuracy. Originality/value This study presents a new kinematics mapping method, which can realize the complete mapping between the human arm and heterogeneous robotic arm in teleoperation. This method is versatile and can be applied to various mechanical arms with different structures.

show abstract

An integrated vision-based system for efficient robot arm teleoperation

Cited by 8 publications

References 32 publications

A Review of Human–Machine Cooperation in the Robotics Domain

A Review of Human–Machine Cooperation in the Robotics Domain

Master-Slave Arm Heterogeneous Mapping With Link Pose Constraint in Teleoperation System

Human-robot kinematics mapping method based on dynamic equivalent points

Contact Info

Product

Resources

About