Towards an intuitive human-robot interaction based on hand gesture recognition and proximity sensors

Al, Gorkem Anil; Estrela, Pedro; Martinez-Hernandez, Uriel

doi:10.1109/mfi49285.2020.9235264

Cited by 18 publications

(8 citation statements)

References 18 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It makes then possible to create sign sequences that could be used in the case of assistive technologies or to control machines. Among the possible applications, we could for example consider driving a wheelchair or even easier professional integration of people with disabilities in an Augmented or Virtual Reality environment or even human-robot collaboration [48].…”

Section: Conclusion and Prospectsmentioning

confidence: 99%

Towards a GUI Gesture Control Using the Leap Motion Controller

Vannobel¹,

Bekaert²,

Baumann³

2022

MMC_C

View full text Add to dashboard Cite

The Leap Motion Controller is a small hand motion capture device that has been the subject of numerous studies since it was launched in 2013. This sensor is particularly used in the field of disability, diagnosis or functional rehabilitation. This article presents two preliminary studies on the use of this device as a gesture mouse and then as a gesture controller. If the Leap Motion Controller is simple to implement, its lack of precision in some gesture configurations and of reliability (outliers) is a barrier which could certainly be lifted in the context of a multimodal interaction.

show abstract

Section: Conclusion and Prospectsmentioning

confidence: 99%

Towards a GUI Gesture Control Using the Leap Motion Controller

Vannobel¹,

Bekaert²,

Baumann³

2022

MMC_C

View full text Add to dashboard Cite

show abstract

“…The tracking of the operator’s hand is achieved by employing a Leap Motion sensor, in which a Kalman filter is used for the position estimation while the orientation is computed by a particle filter. A similar contactless hand gesture recognition system is presented in [ 70 ] for safe HRI. This multi-modal sensor interface uses proximity and gesture sensors, and it can identify real-time hand gestures to control the robot platform.…”

Section: Robotic Systems and Human-robot Perceptionmentioning

confidence: 99%

Human-Robot Perception in Industrial Environments: A Survey

Bonci

Cheng

Indri

et al. 2021

Sensors

View full text Add to dashboard Cite

Perception capability assumes significant importance for human–robot interaction. The forthcoming industrial environments will require a high level of automation to be flexible and adaptive enough to comply with the increasingly faster and low-cost market demands. Autonomous and collaborative robots able to adapt to varying and dynamic conditions of the environment, including the presence of human beings, will have an ever-greater role in this context. However, if the robot is not aware of the human position and intention, a shared workspace between robots and humans may decrease productivity and lead to human safety issues. This paper presents a survey on sensory equipment useful for human detection and action recognition in industrial environments. An overview of different sensors and perception techniques is presented. Various types of robotic systems commonly used in industry, such as fixed-base manipulators, collaborative robots, mobile robots and mobile manipulators, are considered, analyzing the most useful sensors and methods to perceive and react to the presence of human operators in industrial cooperative and collaborative applications. The paper also introduces two proofs of concept, developed by the authors for future collaborative robotic applications that benefit from enhanced capabilities of human perception and interaction. The first one concerns fixed-base collaborative robots, and proposes a solution for human safety in tasks requiring human collision avoidance or moving obstacles detection. The second one proposes a collaborative behavior implementable upon autonomous mobile robots, pursuing assigned tasks within an industrial space shared with human operators.

show abstract

“…These robots need to be capable of physically sensing and perceiving their surrounding environment to make reliable decisions, actions and avoid collisions that can harm humans and other robots [3]. Tactile sensing is an essential component that needs to be considered in the design and development of robots to allow them to physically explore their surrounding environment for safe interaction with humans and other robots, detection of contact, recognition and manipulation of objects [4], [5].…”

Section: Introductionmentioning

confidence: 99%

Multimodal Barometric and Inertial Measurement Unit-Based Tactile Sensor for Robot Control

Martinez-Hernandez

2023

IEEE Sensors J.

Self Cite

View full text Add to dashboard Cite

In this paper, we present a low-cost multimodal tactile sensor capable of providing accelerometer, gyroscope and pressure data using a 7-axis chip as sensing element. This approach reduces the complexity of tactile sensor design and collection of multimodal data. The tactile device is composed of a top layer (a printed circuit board and sensing element), a middle layer (soft rubber material), and bottom layer (plastic base) forming a sandwich structure. This approach allows the measurement of multimodal data when force is applied on different parts of the top layer of the sensor. The multimodal tactile sensor is validated with analyses and experiments in both offline and real-time. First, the spatial impulse response and sensitivity of the sensor are analysed with accelerometer, gyroscope and pressure data systematically collected from the sensor. Second, estimation of contact location from a range of sensor positions and force values is evaluated using accelerometer and gyroscope data together with a Convolutional Neural Network (CNN) method. Third, the estimation of contact location is used to control the position of a robot arm. The results show that the proposed multimodal tactile sensor has the potential for robotic applications such as tactile perception for robot control, human-robot interaction and object exploration.

show abstract

Towards an intuitive human-robot interaction based on hand gesture recognition and proximity sensors

Cited by 18 publications

References 18 publications

Towards a GUI Gesture Control Using the Leap Motion Controller

Towards a GUI Gesture Control Using the Leap Motion Controller

Human-Robot Perception in Industrial Environments: A Survey

Multimodal Barometric and Inertial Measurement Unit-Based Tactile Sensor for Robot Control

Contact Info

Product

Resources

About