Human-Drone Interaction: Using Pointing Gesture to Define a Target Object

Medeiros, Anna C. S.; Ratsamee, Photchara; Uranishi, Yuki; Mashita, Tomohiro; Takemura, Haruo

doi:10.1007/978-3-030-49062-1_48

Cited by 13 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the wide variety of applications and multiple degrees of freedom, it is challenging to design a natural interaction, namely, Human-Drone Interaction, to control UAVs effectively. In past works, we analyzed which gesture fits best for a particular scenario, used our Gesture Development Process [5], validated it in general tasks and with a diverse group of users [6], and conducted an elicitation study with the targeted users [7].…”

Section: Resultsmentioning

confidence: 99%

“…In the present work, we use the gesture for "Select" in a very specific domain of application: as a collaboration interface between UAVs and Firefighters. To doublecheck that the same gesture could be used for "Select" in this particular environment, we performed an elicitation study [7] with seven Firefighters.…”

Section: Gesture Development Processmentioning

confidence: 99%

“…This approach was first presented in [7], where object detection and pose estimation were combined to decide which object is selected by the user. For Pose Estimation, we used OpenPOSE [25].…”

Section: D Pointing Gesture Interface Approachmentioning

confidence: 99%

“…The process of associating a gesture to a particular action or function of the system is not trivial, because it must take into account several factors such as ergonomics, intuitiveness, and objectivity [3,4]. In past works, we analyzed which gesture fits best the purposed scenario, using our Gesture Development Process [5], validating it in general tasks and with a diverse group of users [6], and an elicitation study with the targeted users [7].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

3D Pointing Gestures as Target Selection Tools for Monocular UAVs

Medeiros¹,

Ratsamee²,

Orlosky³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Firefighters need to gain information from both inside and outside of buildings in first response emergency scenarios. For this purpose, drones are beneficial. This paper presents an elicitation study that showed the firefighters' desire to collaborate with autonomous drones. We developed a Human-Drone Interaction (HDI) method for indicating a target to a drone using 3D pointing gestures estimated solely from a monocular camera. The participant first points to a window without using any wearable or body-attached device. Through the drone's front-facing camera, the drone detects the gesture and computes the target window. This work includes a description of the process for choosing the gesture, detecting and localizing objects, and carrying out the transformations between coordinate systems. Our proposed 3D pointing gesture improves a 2D pointing gesture interface by integrating depth information with SLAM, solving multiple objects aligned on the same plane ambiguity, in a large-scale outdoor environment. Experimental results showed that our 3D pointing gesture interface obtained a 0.85 and 0.73 F1-Score on average in simulation and real-world experiments and 0.58 F1-Score at the maximum distance of 25 meters between drone and building.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Gesture Development Processmentioning

confidence: 99%

Section: D Pointing Gesture Interface Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

3D Pointing Gestures as Target Selection Tools for Monocular UAVs

Medeiros¹,

Ratsamee²,

Orlosky³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the present work, we use the gesture for "Select" previously presented in [5] in a very specific domain of application: as a collaboration interface between UAVs and Firefighters. To double-check that the same gesture could be used for "Select" in this particular environment, we performed an elicitation study [7] with seven Firefighters.…”

Section: Defining the Pointing Gesturementioning

confidence: 99%

3D Pointing Gestures as Target Selection Tools: Guiding Monocular UAVs during Window Selection in an Outdoor Environment

Medeiros

Ratsamee

Orlosky

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Firefighters need to gain information from both inside and outside of buildings in first response emergency scenarios. For this purpose, drones are beneficial. This paper presents an elicitation study that showed the firefighters’ desire to collaborate with autonomous drones. We developed a Human-Drone Interaction (HDI) method for indicating a target to a drone using 3D pointing gestures estimated solely from a monocular camera. The participant first points to a window without using any wearable or body-attached device. Through the drone’s front-facing camera, the drone detects the gesture and computes the target window. This work includes a description of the process for choosing the gesture, detecting and localizing objects, and carrying out the transformations between coordinate systems. Our proposed 3D pointing gesture interface improves a 2D pointing gesture interface by integrating depth information with SLAM, solving multiple objects aligned on the same plane ambiguity, in a large-scale outdoor environment. Experimental results showed that our 3D pointing gesture interface obtained a 0.85 and 0.73 F1-Score on average in simulation and real-world experiments and 0.58 F1-Score at the maximum distance of 25 meters between drone and building.

show abstract