Applying automatic speech recognition technology to Air Traffic Management

Abstract

Controller-pilot voice communications are a critical component of the Air Traffic Control (ATC) system, but outside of the human listening and responding that occurs with each transmission, they are an underutilized source of information for automation systems in the ATC domain. Automatic speech recognition is a continuously improving technology that can be used to tap into this information source for potential system benefits in a variety of ATC applications, such as monitoring live operations for safety benefit, conducting analysis on large quantities of recorded controller-pilot speech, or enabling automated simulation pilots to facilitate training and Human-in-the-Loop (HITL) simulation experiments. This paper describes how automatic speech recognition can be used in the ATC domain, the characteristics of the automatic speech recognition process and the ATC domain that make the problem unique, and the engineering process for effectively applying automatic speech recognition to ATC systems.Some of the information transmitted by voice communications is manually entered into automation systems in particular situations (e.g., a pilot entering a clearance into a flight management system or an en route controller entering an interim altitude in the Host Computer System), but for the most part, voice communication information is not captured in a way that is immediately text searchable or analyzable without intermediary human interpretation and processing. Controller-pilot voice communication, therefore, is an underutilized source of information within the ATC system, and automatic speech recognition can help unlock this potentially valuable information source.

Natural Language Processing (NLP) in Aviation Safety: Systematic Review of Research and Outlook into the Future

“…ASR shows great potential in developing automatic safety-critical systems in aviation operations, such as the detection of communication incurred by human errors, deviations from voice instructions, and operator status monitoring [61]. Six out of ten selected studies explored ASR and NLP for safety support from different perspectives; automated ATC communication error detection to prevent loss of separation, aviation radiotelephony readback verification, operator cognitive functions load estimation, and speech emotion recognition are examples [16,17,42,43,45,46].…”

Design and Evaluation of the Closed Runway Operation Prevention Device

“…Automatic speech recognition systems match patterns of digitized audio of spoken words against computer models of known speech to "recognize" the spoken words or phrases (Chen et al, 2012). Whether or not a verbatim transcription of the speech is needed, however, depends on the particular application of the speech recognition system (Kopald, Chanen, et al, 2013). For example, in some cases, the system may only need to identify the presence of a particular word or phrase, whereas in others it may need to recognize more content to decipher the speaker's overall intent.…”

“…However, nonstandard phraseology deviations, fast pace (cadence), slurring, and accents in controller speech, as well as acoustic distortions introduced by the ATC environment and voice switching equipment, complicate the speech recognition task. The limited population of speakers and the application of various speech recognition tuning techniques can help mitigate these challenges (Kopald, Chanen, et al, 2013).…”

Improving speech recognition models with small samples for air traffic control systems

“…In practice, however, such HITL procedure is considered to present a safety risk and thus to be in need of monitoring using advanced techniques. It is believed that understanding the spoken instructions is an efficient way to monitor the HITL risk and further formulate a closed-loop ATC management system [2]. To this end, automatic speech recognition (ASR) is a powerful interface for human-machine interaction that can allow a machine to automatically understand real-time ATC speech conversations to support further applications.…”