Semi-supervised Adaptation of Assistant Based Speech Recognition Models for different Approach Areas

Kleinert, Matthias; Helmke, Hartmut; Siol, Gerald; Ehr, Heiko; Cerna, Aneta; Kern, Christian; Klakow, Dietrich; Motlíček, Petr; Oualil, Youssef; Singh, Mittul; Srinivasamurthy, Ajay

doi:10.1109/dasc.2018.8569879

Cited by 19 publications

(19 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, spoken ATC communications from LiveATC.net were collected and transcribed; LiveATC uses similar VHF receivers as the ones that are intended for use. Currently, the call sign detection performance is approximately six times worse than what MALORCA has demonstrated [7,8] (although the contextual information is so far avoided). We attribute this to the low (signal-to-noise) ratio of the collected speech segments.…”

Section: Motivationmentioning

confidence: 88%

Automatic Call Sign Detection: Matching Air Surveillance Data with Air Traffic Spoken Communications

Zuluaga-Gómez

Veselý

Blatt

et al. 2020

8th OpenSky Symposium 2020

Self Cite

View full text Add to dashboard Cite

Voice communication is the main channel to exchange information between pilots and Air-Traffic Controllers (ATCos). Recently, several projects have explored the employment of speech recognition technology to automatically extract spoken key information such as call signs, commands, and values, which can be used to reduce ATCos’ workload and increase performance and safety in Air-Traffic Control (ATC)-related activities. Nevertheless, the collection of ATC speech data is very demanding, expensive, and limited to the intrinsic speakers’ characteristics. As a solution, this paper presents ATCO2, a project that aims to develop a unique platform to collect, organize, and pre-process ATC data collected from air space. Initially, the data are gathered directly through publicly accessible radio frequency channels with VHF receivers and LiveATC, which can be considered as an “unlimited-source” of low-quality data. The ATCO2 project explores employing context information such as radar and air surveillance data (collected with ADS-B and Mode S) from the OpenSky Network (OSN) to correlate call signs automatically extracted from voice communication with those available from ADS-B channels, to eventually increase the overall call sign detection rates. More specifically, the timestamp and location of the spoken command (issued by the ATCo by voice) are extracted, and a query is sent to the OSN server to retrieve the call sign tags in ICAO format for the airplanes corresponding to the given area. Then, a word sequence provided by an automatic speech recognition system is fed into a Natural Language Processing (NLP) based module together with the set of call signs available from the ADS-B channels. The NLP module extracts the call sign, command, and command arguments from the spoken utterance.

show abstract

Section: Motivationmentioning

confidence: 88%

Automatic Call Sign Detection: Matching Air Surveillance Data with Air Traffic Spoken Communications

Zuluaga-Gómez

Veselý

Blatt

et al. 2020

8th OpenSky Symposium 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition to the well-known language assistants from Apple (Siri), Amazon (Alexa), Samsung (S Voice) and the Google assistant, there exist also less common assistants, such as Cortana from Microsoft and Facebook's M [14]. IVA platforms apply speech recognition as input interface of their speech assistants [15]. This involves direct human-machine communication, which triggers a defined action by sending a command to the speech assistant [16].…”

Section: Integrated Voice Assistantsmentioning

confidence: 99%

Adoption of Integrated Voice Assistants in Health Care – Requirements and Design Guidelines

Eggert¹,

Stanke²

2020

WI2020 Zentrale Tracks

View full text Add to dashboard Cite

Integrated voice assistants (IVA) receive more and more attention and are widespread for entertainment use cases, such as radio hearing or web searches. At the same time, the health care segment suffers in process inefficiency and missing staff, whereas the usage of IVA has the potential to improve caring processes and patient satisfaction. By applying a design science approach and based on a qualitative study, we identify IVA requirements, barriers and design guidelines for the health care sector. The results reveal three important IVA functions: the ability to set appointments with care service staff, the documentation of health history and the communication with service staff. Integration, system stability and volume control are the most important nonfunctional requirements. Based on the interview results and project experiences, six design and implementation guidelines are derived.

show abstract

“…every few seconds) of the airspace and hence can be used to provide a situational context for the ASR engine. Given a dataset of airspace situations (encoded in the radar data) and the corresponding ground-truth commands issued by controllers, we build a Command Prediction Model (CPM) [22,23] that provides a list of plausible commands for a given airspace situation. This list of commands is called the dynamic context and is used for iterative semi-supervised learning.…”

Section: Command Prediction Modelmentioning

confidence: 99%

Iterative Learning of Speech Recognition Models for Air Traffic Control

et al. 2018

Self Cite

View full text Add to dashboard Cite

Automatic Speech Recognition (ASR) has recently proved to be a useful tool to reduce the workload of air traffic controllers leading to significant gains in operational efficiency. Air Traffic Control (ATC) systems in operation rooms around the world generate large amounts of untranscribed speech and radar data each day, which can be utilized to build and improve ASR models. In this paper, we propose an iterative approach that utilizes increasing amounts of untranscribed data to incrementally build the necessary ASR models for an ATC operational area. Our approach uses a semi-supervised learning framework to combine speech and radar data to iteratively update the acoustic model, language model and command prediction model (i.e. prediction of possible commands from radar data for a given air traffic situation) of an ASR system. Starting with seed models built with a limited amount of manually transcribed data, we simulate an operational scenario to adapt and improve the models through semi-supervised learning. Experiments on two independent ATC areas (Vienna and Prague) demonstrate the utility of our proposed methodology that can scale to operational environments with minimal manual effort for learning and adaptation.

show abstract

Semi-supervised Adaptation of Assistant Based Speech Recognition Models for different Approach Areas

Cited by 19 publications

References 17 publications

Automatic Call Sign Detection: Matching Air Surveillance Data with Air Traffic Spoken Communications

Automatic Call Sign Detection: Matching Air Surveillance Data with Air Traffic Spoken Communications

Adoption of Integrated Voice Assistants in Health Care – Requirements and Design Guidelines

Iterative Learning of Speech Recognition Models for Air Traffic Control

Contact Info

Product

Resources

About