A Gesture-Based Teleoperation System for Compliant Robot Motion

Zhang, Wei; Cheng, Hongtai; Zhao, Liang; Hao, Lina; Tao, Manli; Xiang, Chaoqun

doi:10.3390/app9245290

Cited by 20 publications

(7 citation statements)

References 29 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The current systems that utilize gestures to communicate with a human mainly focus on the teleoperation of the robot (e.g., [5]) or use gestures to command the robot without the option to connect the human gestures to the current state of the scene (e.g., [6], [7], [8]). Zhang et al [5] uses similarly to us Leap motion sensor to detect hand movements and classify individual gestures. A basic set of gestures is used to teleoperate the robotic arm without connection to the given scene.…”

Section: Related Workmentioning

confidence: 99%

Communicating human intent to a robotic companion by multi-type gesture sentences

Petr¹,

Behrens²,

Štěpánová³

et al. 2023

Preprint

View full text Add to dashboard Cite

Human-Robot collaboration in home and industrial workspaces is on the rise. However, the communication between robots and humans is a bottleneck. Although people use a combination of different types of gestures to complement speech, only a few robotic systems utilize gestures for communication. In this paper, we propose a gesture pseudolanguage and show how multiple types of gestures can be combined to express human intent to a robot (i.e., expressing both the desired action and its parameters -e.g., pointing to an object and showing that the object should be emptied into a bowl). The demonstrated gestures and the perceived tabletop scene (object poses detected by CosyPose) are processed in real-time) to extract the human's intent. We utilize behavior trees to generate reactive robot behavior that handles various possible states of the world (e.g., a drawer has to be opened before an object is placed into it) and recovers from errors (e.g., when the scene changes). Furthermore, our system enables switching between direct teleoperation of the end-effector and high-level operation using the proposed gesture sentences. The system is evaluated on increasingly complex tasks using a real 7-DoF Franka Emika Panda manipulator. Controlling the robot via action gestures lowered the execution time by up to 60%, compared to direct teleoperation.

show abstract

Section: Related Workmentioning

confidence: 99%

Communicating human intent to a robotic companion by multi-type gesture sentences

Petr¹,

Behrens²,

Štěpánová³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The issue of motion control is critical in teleoperation robotic systems. To overcome the limitations of human motion accuracy, a paper [18] introduces new interaction logic, a scalable human-robot motion mapping mechanism and a single axis mode to balance teleoperation efficiency and accuracy. In order to meet the requirements of complaint assembly skill, a vibration-based force feedback system was developed to let the operator feel the contact force.…”

Section: Motion Controlmentioning

confidence: 99%

Optimization of Motion Planning and Control for Automatic Machines, Robots and Multibody Systems

Boscariol¹,

Richiedei²

2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…It is quite important for healthcare directors and policy makers [31]. It is also as much a concern for patients and operators who must operationalize them outside formal clinical situations [32,33]. It is worth mentioning that mobile robots are often considered the best alternative to explore and implement in aggressive remote areas.…”

Section: Introductionmentioning

confidence: 99%

Orientation Control Design of a Telepresence Robot: An Experimental Verification in Healthcare System

et al. 2023

View full text Add to dashboard Cite

Automation in the modern world has become a necessity for humans. Intelligent mobile robots have become necessary to perform various complex tasks in healthcare and industry environments. Mobile robots have gained attention during the pandemic; human–robot interaction has become vibrant. However, there are many challenges in obtaining human–robot interactions regarding maneuverability, controllability, stability, drive layout and autonomy. In this paper, we proposed a stability and control design for a telepresence robot called auto-MERLIN. The proposed design simulated and experimentally verified self-localization and maneuverability in a hazardous environment. A model from Rieckert and Schunck was initially considered to design the control system parameters. The system identification approach was then used to derive the mathematical relationship between the manipulated variable of robot orientation control. The theoretical model of the robot mechanics and associated control were developed. A design model was successfully implemented, analyzed mathematically, used to build the hardware and tested experimentally. Each level takes on excellent tasks for the development of auto-MERLIN. A higher level always uses the services of lower levels to carry out its functions. The proposed approach is comparatively simple, less expensive and easily deployable compared to previous methods. The experimental results showed that the robot is functionally complete in all aspects. A test drive was performed over a given path to evaluate the hardware, and the results were presented. Simulation and experimental results showed that the target path is maintained quite well.

show abstract

A Gesture-Based Teleoperation System for Compliant Robot Motion

Cited by 20 publications

References 29 publications

Communicating human intent to a robotic companion by multi-type gesture sentences

Communicating human intent to a robotic companion by multi-type gesture sentences

Optimization of Motion Planning and Control for Automatic Machines, Robots and Multibody Systems

Orientation Control Design of a Telepresence Robot: An Experimental Verification in Healthcare System

Contact Info

Product

Resources

About