Natural Language Based Multimodal Interface for UAV Mission Planning

Chandarana, Meghan; Meszaros, Erica L.; Trujillo, Anna C.; Allen, B. Danette

doi:10.1177/1541931213601483

Cited by 13 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other alternative interfaces that rely on motion tracking devices, such as the leap motion controller, can detect gestures such as forearm supination, thereby providing hands-free steering wheel rotation (Akyol and Canzler, 2000; Chandarana et al , 2017). As in the case of joysticks, however, the sensor for the motion tracking device may be installed in a location that adds more distance between the hands of the driver and the steering wheel, in comparison to the sEMG-based interface.…”

Section: Design Of the Steering Assistance Systemmentioning

confidence: 99%

A surface electromyography controlled steering assistance interface

Nacpil

Zheng

Kaizuka

et al. 2019

JICV

View full text Add to dashboard Cite

Purpose Two-handed automobile steering at low vehicle speeds may lead to reduced steering ability at large steering wheel angles and shoulder injury at high steering wheel rates (SWRs). As a first step toward solving these problems, this study aims, firstly, to design a surface electromyography (sEMG) controlled steering assistance interface that enables hands-free steering wheel rotation and, secondly, to validate the effect of this rotation on path-following accuracy. Design/methodology/approach A total of 24 drivers used biceps brachii sEMG signals to control the steering assistance interface at a maximized SWR in three driving simulator scenarios: U-turn, 90º turn and 45º turn. For comparison, the scenarios were repeated with a slower SWR and a game steering wheel in place of the steering assistance interface. The path-following accuracy of the steering assistance interface would be validated if it was at least comparable to that of the game steering wheel. Findings Overall, the steering assistance interface with a maximized SWR was comparable to a game steering wheel. For the U-turn, 90º turn and 45º turn, the sEMG-based human–machine interface (HMI) had median lateral errors of 0.55, 0.3 and 0.2 m, respectively, whereas the game steering wheel, respectively, had median lateral errors of 0.7, 0.4 and 0.3 m. The higher accuracy of the sEMG-based HMI was statistically significant in the case of the U-turn. Originality/value Although production automobiles do not use sEMG-based HMIs, and few studies have proposed sEMG controlled steering, the results of the current study warrant further development of a sEMG-based HMI for an actual automobile.

show abstract

Section: Design Of the Steering Assistance Systemmentioning

confidence: 99%

A surface electromyography controlled steering assistance interface

Nacpil

Zheng

Kaizuka

et al. 2019

JICV

View full text Add to dashboard Cite

show abstract

“…Gesture-based drone interfaces have been actively researched (e.g., [20,52,58]) to make drone control more intuitive and fun. Voice-based control has also been introduced [8,9,48]. These interfaces are intuitive and allow segments of the drone movement paths to be input.…”

Section: Ar-based Egocentric Drone Interfacementioning

confidence: 99%

PinpointFly: An Egocentric Position-control Drone Interface using Mobile AR

Chen

Takashima

Fujita

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Accurate drone positioning is challenging because pilots only have a limited position and direction perception of a fying drone from their perspective. This makes conventional joystick-based speed control inaccurate and more complicated and signifcantly degrades piloting performance. We propose PinpointFly, an egocentric drone interface that allows pilots to arbitrarily position and rotate a drone using position-control direct interactions on a see-through mobile AR where the drone position and direction are visualized with a virtual cast shadow (i.e., the drone's orthogonal projection onto the foor). Pilots can point to the next position or draw the drone's fight trajectory by manipulating the virtual cast shadow and the direction/height slider bar on the touchscreen. We design and implement a prototype of PinpointFly for indoor and visual line of sight scenarios, which are comprised of real-time and predefned motion-control techniques. We conduct two user studies with simple positioning and inspection tasks. Our results demonstrate that PinpointFly makes the drone positioning and inspection operations faster, more accurate, simpler and fewer workload than a conventional joystick interface with a speed-control method. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI).

show abstract

“…Chandarana et al [35] presented a custom-developed software using speech and gesture recognition for UAV path planning. In their research, the authors performed a comparison of natural language interfaces using a mouse-based interface as a baseline and evaluating individual users who were required to complete a flight path composed of flight trajectories or actions.…”

Section: Speech Controlmentioning

confidence: 99%

Unmanned Aerial Vehicle Control through Domain-Based Automatic Speech Recognition

2020

View full text Add to dashboard Cite

Currently, unmanned aerial vehicles, such as drones, are becoming a part of our lives and extend to many areas of society, including the industrialized world. A common alternative for controlling the movements and actions of the drone is through unwired tactile interfaces, for which different remote control devices are used. However, control through such devices is not a natural, human-like communication interface, which sometimes is difficult to master for some users. In this research, we experimented with a domain-based speech recognition architecture to effectively control an unmanned aerial vehicle such as a drone. The drone control was performed in a more natural, human-like way to communicate the instructions. Moreover, we implemented an algorithm for command interpretation using both Spanish and English languages, as well as to control the movements of the drone in a simulated domestic environment. We conducted experiments involving participants giving voice commands to the drone in both languages in order to compare the effectiveness of each, considering the mother tongue of the participants in the experiment. Additionally, different levels of distortion were applied to the voice commands to test the proposed approach when it encountered noisy input signals. The results obtained showed that the unmanned aerial vehicle was capable of interpreting user voice instructions. Speech-to-action recognition improved for both languages with phoneme matching in comparison to only using the cloud-based algorithm without domain-based instructions. Using raw audio inputs, the cloud-based approach achieves 74.81% and 97.04% accuracy for English and Spanish instructions, respectively. However, with our phoneme matching approach the results are improved, yielding 93.33% accuracy for English and 100.00% accuracy for Spanish.

show abstract

Natural Language Based Multimodal Interface for UAV Mission Planning

Cited by 13 publications

References 15 publications

A surface electromyography controlled steering assistance interface

A surface electromyography controlled steering assistance interface

PinpointFly: An Egocentric Position-control Drone Interface using Mobile AR

Unmanned Aerial Vehicle Control through Domain-Based Automatic Speech Recognition

Contact Info

Product

Resources

About