Foundations of Visual Linear Human–Robot Interaction via Pointing Gesture Navigation

Tölgyessy, Michal; Dekan, Martin; Duchoň, František; Rodina, Jozef; Hubinský, Peter; Chovanec, Ľuboš

doi:10.1007/s12369-017-0408-9

Cited by 45 publications

(34 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Proximity interaction techniques can take advantage of pointing gestures to intuitively express locations or objects with minimal cognitive overhead; this modality has been often used in HRI research e.g. for pick‐and‐place tasks (Brooks & Breazeal, ; Cosgun, Trevor, & Christensen, ; Droeschel, Stückler, & Behnke, ; Großmann et al, ), labeling and/or querying information about objects or locations (Akkil & Isokoski, ; Brooks & Breazeal, ; Pateraki, Baltzakis, & Trahanias, ), selecting a robot within a group (Nagi, Giusti, Gambardella, & Di Caro, ; Pourmehr, Monajjemi, Wawerla, Vaughan, & Mori, ), and providing navigational goals (Raza Abidi, Williams, & Johnston, ; Gromov et al, , ; Jevtić, Doisy, Parmet, & Edan, ; Tölgyessy et al, ; Van den Bergh et al, ; Wolf et al, ). Such gestures can enable rescue workers to easily direct multiple robots, and robot types, using the same interface (see Figure ).…”

Section: State Of the Artmentioning

confidence: 99%

The current state and future outlook of rescue robotics

Delmerico

Mintchev

Giusti

et al. 2019

Journal of Field Robotics

222

124

View full text Add to dashboard Cite

Robotic technologies, whether they are remotely operated vehicles, autonomous agents, assistive devices, or novel control interfaces, offer many promising capabilities for deployment in real-world environments. Postdisaster scenarios are a particularly relevant target for applying such technologies, due to the challenging conditions faced by rescue workers and the possibility to increase their efficacy while decreasing the risks they face. However, field-deployable technologies for rescue work have requirements for robustness, speed, versatility, and ease of use that may not be matched by the state of the art in robotics research. This paper aims to survey How to cite this article: Delmerico J, Mintchev S, Giusti A, et al. The current state and future outlook of rescue robotics.

show abstract

Section: State Of the Artmentioning

confidence: 99%

The current state and future outlook of rescue robotics

Delmerico

Mintchev

Giusti

et al. 2019

Journal of Field Robotics

222

124

View full text Add to dashboard Cite

show abstract

“…The work of Tölgyessy [20] investigates the use of pointing gestures to give target locations to Roomba-like robots. It uses an RGBD sensor (Kinect) mounted on a 2-DoF robot to interpret the pointing gesture from a person standing 2-3 meters from the setup.…”

Section: Related Workmentioning

confidence: 99%

“…There are multiple studies that research the use of gestures to navigate a UAV [23,20,16,22]. However, we investigate the collaboration between UAVs and humans to relieve human task load and reduce UAV's error by processing periodical human-input.…”

Section: Related Workmentioning

confidence: 99%

3D Pointing Gestures as Target Selection Tools for Monocular UAVs

Medeiros¹,

Ratsamee²,

Orlosky³

et al. 2020

Preprint

View full text Add to dashboard Cite

Firefighters need to gain information from both inside and outside of buildings in first response emergency scenarios. For this purpose, drones are beneficial. This paper presents an elicitation study that showed the firefighters' desire to collaborate with autonomous drones. We developed a Human-Drone Interaction (HDI) method for indicating a target to a drone using 3D pointing gestures estimated solely from a monocular camera. The participant first points to a window without using any wearable or body-attached device. Through the drone's front-facing camera, the drone detects the gesture and computes the target window. This work includes a description of the process for choosing the gesture, detecting and localizing objects, and carrying out the transformations between coordinate systems. Our proposed 3D pointing gesture improves a 2D pointing gesture interface by integrating depth information with SLAM, solving multiple objects aligned on the same plane ambiguity, in a large-scale outdoor environment. Experimental results showed that our 3D pointing gesture interface obtained a 0.85 and 0.73 F1-Score on average in simulation and real-world experiments and 0.58 F1-Score at the maximum distance of 25 meters between drone and building.

show abstract

“…They allow a person to intuitively and efficiently communicate locations and other spatial notions (trajectories, directions). Research in robotics has used pointing gestures for disparate tasks: pick-and-place [7,8,9], object and area labeling [10], teaching by demonstration [11], point-to-goal [12,13,14], selection of a robot within a group [15,16], and assessment of the joint attention [17]. Other works use non-pointing gestures for interacting with co-located robots, e.g.…”

Section: Related Workmentioning

confidence: 99%

Proximity Human-Robot Interaction Using Pointing Gestures and a Wrist-mounted IMU

Gromov

Abbate

Gambardella

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We present a system for interaction between colocated humans and mobile robots, which uses pointing gestures sensed by a wrist-mounted IMU. The operator begins by pointing, for a short time, at a moving robot. The system thus simultaneously determines: that the operator wants to interact; the robot they want to interact with; and the relative pose among the two. Then, the system can reconstruct pointed locations in the robot's own reference frame, and provide real-time feedback about them so that the user can adapt to misalignments. We discuss the challenges to be solved to implement such a system and propose practical solutions, including variants for fast flying robots and slow ground robots. We report different experiments with real robots and untrained users, validating the individual components and the system as a whole.

show abstract

Foundations of Visual Linear Human–Robot Interaction via Pointing Gesture Navigation

Cited by 45 publications

References 25 publications

The current state and future outlook of rescue robotics

The current state and future outlook of rescue robotics

3D Pointing Gestures as Target Selection Tools for Monocular UAVs

Proximity Human-Robot Interaction Using Pointing Gestures and a Wrist-mounted IMU

Contact Info

Product

Resources

About