Unmanned Aerial Vehicle Control through Domain-Based Automatic Speech Recognition

Abstract

Currently, unmanned aerial vehicles, such as drones, are becoming a part of our lives and extend to many areas of society, including the industrialized world. A common alternative for controlling the movements and actions of the drone is through unwired tactile interfaces, for which different remote control devices are used. However, control through such devices is not a natural, human-like communication interface, which sometimes is difficult to master for some users. In this research, we experimented with a domain-based speech recognition architecture to effectively control an unmanned aerial vehicle such as a drone. The drone control was performed in a more natural, human-like way to communicate the instructions. Moreover, we implemented an algorithm for command interpretation using both Spanish and English languages, as well as to control the movements of the drone in a simulated domestic environment. We conducted experiments involving participants giving voice commands to the drone in both languages in order to compare the effectiveness of each, considering the mother tongue of the participants in the experiment. Additionally, different levels of distortion were applied to the voice commands to test the proposed approach when it encountered noisy input signals. The results obtained showed that the unmanned aerial vehicle was capable of interpreting user voice instructions. Speech-to-action recognition improved for both languages with phoneme matching in comparison to only using the cloud-based algorithm without domain-based instructions. Using raw audio inputs, the cloud-based approach achieves 74.81% and 97.04% accuracy for English and Spanish instructions, respectively. However, with our phoneme matching approach the results are improved, yielding 93.33% accuracy for English and 100.00% accuracy for Spanish.

Flying Robot Technology (Drone) Trends: A Review in the Building and Construction Industry

“…Body shapes, body materials, and different components make up the main mechanism of a drone [41]. Drone technology has a frame structure, propeller, motor, power system, electronic control, and communication system [42]. In addition, a camera or thermal infrared camera can be attached to record video during flight [43].…”

Persistent Rule-based Interactive Reinforcement Learning

“…When teaching or conveying information between people, one form of knowledge transfer is rules. While the syntax of the rule is not necessarily formal, the relation of condition and conclusion is maintained [20]. For example, the sentence 'Don't touch the stove when hot' can be represented formally as 'IF stove==hot THEN don't touch'.…”

“…Moreover, conditions and conclusions are quickly identifiable by humans when using natural language is used. Recent advances in speech-to-text systems have demonstrated the ability to identify the condition and conclusion in human speech [21,20]. Assistant systems such as Google Assistant and Apple Siri have demonstrated this functionality through conditional reminders such as 'When I am at the supermarket remind me to get milk' or 'Remind me at 3 PM to call the restaurant' [22].…”

Evaluation of Human Performance in the Operation of a UAV in a Joint Operation Scenario with Troops on the Ground

“…The experiment is divided into two rounds; in one, the volunteer will act only as a Sensor Operator (SO), being responsible for detecting and identifying targets in an area of interest, and in the other, he/she will accumulate the role of Pilot (1P), having to command the aircraft according to the mission control guidelines. When the volunteer occupies the role of 1P, he can use the fully manual interface or voice commands for the entire mission to navigate the aircraft (Contreras et al (2020)). The objectives of the assessment will be to identify the variation in performance resulting from the accumulation of functions and to evaluate the impact of an alternative HMI on the conduct of the mission.…”