Towards Real-Time Physical Human-Robot Interaction Using Skeleton Information and Hand Gestures

Mazhar, Osama; Ramdani, Sofiane; Navarro, Benjamin; Passama, Robin; Cherubini, Andrea

doi:10.1109/iros.2018.8594385

Cited by 43 publications

(22 citation statements)

References 15 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of multiple GPUs for OpenPose library can enhance the temporal performance of our system. We explain our presented work in more detail in [10]. We plan to extend our work by developing a background independent hand gesture detector by substituting backgrounds with rich-textured images.…”

Section: Phri Experiments and Resultsmentioning

confidence: 99%

A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

Mazhar

Ramdani

Navarro

et al. 2018

2018 IEEE Workshop on Advanced Robotics and Its Social Impacts (ARSO)

Self Cite

View full text Add to dashboard Cite

Section: Phri Experiments and Resultsmentioning

confidence: 99%

A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

Mazhar

Ramdani

Navarro

et al. 2018

2018 IEEE Workshop on Advanced Robotics and Its Social Impacts (ARSO)

Self Cite

View full text Add to dashboard Cite

“…This paper is an extension of our previous work proposed in [55] which presented a tool handover task between robot and human coworker through static hand gestures. A convolutional neural network, inspired mainly by LeNet [47] was developed, to classify four hand gestures.…”

Section: Our Contributionsmentioning

confidence: 95%

“…We extend our work by training a hand gesture detector on ten gestures instead of four presented in [55]. Moreover, the backgrounds are now replaced with random pattern/indoor-architecture images to make the detection robust and background invariant.…”

Section: Our Contributionsmentioning

confidence: 99%

“…In our previous work [55], we integrated OpenPose with gesture recognition sequentially to obtain an overall temporal resolution of approximately 4 fps. In this work, an inter-process distributed system is designed through nanomsg socket library 1 which has drastically increased the frame rate of the vision pipeline.…”

Section: Asynchronous Integration Of the Modulesmentioning

confidence: 99%

See 1 more Smart Citation

A real-time human-robot interaction framework with robust background invariant hand gesture detection

Mazhar

Navarro

Ramdani

et al. 2019

Robotics and Computer-Integrated Manufacturing

Self Cite

100

View full text Add to dashboard Cite

In the light of factories of the future, to ensure productive and safe interaction between robot and human coworkers, it is imperative that the robot extracts the essential information of the coworker. We address this by designing a reliable framework for real-time safe human-robot collaboration, using static hand gestures and 3D skeleton extraction. OpenPose library is integrated with Microsoft Kinect V2, to obtain a 3D estimation of the human skeleton. With the help of 10 volunteers, we recorded an image dataset of alphanumeric static hand gestures, taken from the American Sign Language. We named our dataset OpenSign and released it to the community for benchmarking. Inception V3 convolutional

show abstract

“…Unlike in our work, in these two works, the adaptation was done at the low-level control of the robot by a hybrid force/impedance controller, while we did it at the symbolic level of the task. A scenario where a human and a robot physically interact through a handover of an object was discussed by Mazhar et al [18]. Force signals were used to identify different phases of the sequence of actions.…”

Section: Related Workmentioning

confidence: 99%

On Inferring Intentions in Shared Tasks for Industrial Collaborative Robots

2019

View full text Add to dashboard Cite

Inferring human operators’ actions in shared collaborative tasks plays a crucial role in enhancing the cognitive capabilities of industrial robots. In all these incipient collaborative robotic applications, humans and robots not only should share space, but also forces and the execution of a task. In this article, we present a robotic system that is able to identify different human’s intentions and to adapt its behavior consequently, only employing force data. In order to accomplish this aim, three major contributions are presented: (a) a force based operator’s intention recognition system based on data from only two users; (b) a force based dataset of physical human–robot interaction; and (c) validation of the whole system with 15 people in a scenario inspired by a realistic industrial application. This work is an important step towards a more natural and user-friendly manner of physical human–robot interaction in scenarios where humans and robots collaborate in the accomplishment of a task.

show abstract

Towards Real-Time Physical Human-Robot Interaction Using Skeleton Information and Hand Gestures

Cited by 43 publications

References 15 publications

A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

A real-time human-robot interaction framework with robust background invariant hand gesture detection

On Inferring Intentions in Shared Tasks for Industrial Collaborative Robots

Contact Info

Product

Resources

About