A Unified Framework for Multilingual Speech Recognition in Air Traffic Control Systems

Lin, Yi; Guo, Dongyue; Zhang, Jianwei; Chen, Zhengmao; Yang, Bo

doi:10.1109/tnnls.2020.3015830

Cited by 46 publications

(45 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When it comes to the ASR research in the ATC domain, our previous works have studied the independent system [1,11] and cascaded multilingual pipeline [2,12]. Airbus held a challenge that focused on translating the ATC speech and detecting the aircraft identification [10].…”

Section: Related Workmentioning

confidence: 99%

“…• The PoV unit is between letter and word from the linguistics perspective. Unlike our previous work [12], all English words can be directly combined from PoV units, which are capable of generating human-readable texts and further formulating an end-to-end ASR paradigm. • By dividing the English word into PoV unit, the class imbalance can be relieved to some extent, as demonstrated in Figure 5.…”

Section: Pronunciation-oriented Vocabularymentioning

confidence: 99%

“…• Non-end-to-end multilingual ASR system (D3): a non-endto-end ASR model is applied to transcribe the Chinese and English speech, simultaneously. The details of this model can be found in [12]. • The proposed approach (D4): The proposed approach, is trained to transcribe the Chinese and English speech, simultaneously, whose configurations are the same as that of the C6.…”

Section: Experiments For Validating the Proposed Asr Modelmentioning

confidence: 99%

“…Although enormous efforts have been made to achieve the ASR task in the ATC domain, such as the independent system for a single language [1,10,11], the cascaded pipeline for multilingual recognition [2,12], it is also believed that an endto-end multilingual framework with the ability to address the above technical problems is the ultimate solution for the ASR research in the ATC domain. To this end, an improved end-toend ASR model is proposed to address the multilingual ASR task in this work, which is able to transcribe the speech signals into human-readable texts in Chinese characters and English words.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Towards multilingual end‐to‐end speech recognition for air traffic control

Lin

Yang

Guo

et al. 2021

IET Intelligent Trans Sys

Self Cite

View full text Add to dashboard Cite

In this work, an end-to-end framework is proposed to achieve multilingual automatic speech recognition (ASR) in air traffic control (ATC) systems. Considering the standard ATC procedure, a recurrent neural network (RNN) based framework is selected to mine the temporal dependencies among speech frames. Facing the distributed feature space caused by the radio transmission, a hybrid feature embedding block is designed to extract high-level representations, in which multiple convolutional neural networks are designed to accommodate different frequency and temporal resolutions. The residual mechanism is performed on the RNN layers to improve the trainability and the convergence. To integrate the multilingual ASR into a single model and relieve the class imbalance, a special vocabulary is designed to unify the pronunciation of the vocabulary in Chinese and English, i.e., pronunciation-oriented vocabulary. The proposed model is optimized by the connectionist temporal classification loss and is validated on a real-world speech corpus (ATC-Speech). A character error rate of 4.4% and 5.9% is achieved for Chinese and English speech, respectively, which outperforms other popular approaches. Most importantly, the proposed approach achieves the multilingual ASR task in an end-to-end manner with considerable high performance. INTRODUCTIONAir traffic control (ATC) is an essential service provided by ground-based air traffic controllers (ATCOs) to guide the flight to be operated in a safe manner (i.e. prevent conflict), and further to organize and expedite the traffic flow. As the primary communication way between the ATCO and the aircrew, the spoken instruction through the very high frequency (VHF) radio transmission implies a wealth of contextualized situational information, which is important to the real-time ATC decisionmaking. In the current ATC management system, the ATC is a non-automatic procedure (human-in-the-loop) and is always regarded as a potential risk for the air traffic operation [1]. Numerous studies have demonstrated that monitoring the control conversation is a promising way to obtain real-time traffic dynamics [2,3,4], which benefits to formulate a closed-loop ATC management. To this end, the automatic speech recognition (ASR) technique, with the purpose of building the bridgeThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Pronunciation-oriented Vocabularymentioning

confidence: 99%

Section: Experiments For Validating the Proposed Asr Modelmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Towards multilingual end‐to‐end speech recognition for air traffic control

Lin

Yang

Guo

et al. 2021

IET Intelligent Trans Sys

Self Cite

View full text Add to dashboard Cite

show abstract

“…The trajectory prediction (TP), as a core technique in air traffic studies, has been attracting more and more attention from all over the world. It is the foundation of performing many air traffic applications, such as traffic flow prediction and conflict detection [1,2]. TP aims at predicting the flight trajectory in the near future based on its flight plan and motion patterns.…”

Section: Introductionmentioning

confidence: 99%

A Deep Gaussian Process-Based Flight Trajectory Prediction Approach and Its Application on Conflict Detection

2020

Self Cite

View full text Add to dashboard Cite

In this work, a deep Gaussian process (DGP) based framework is proposed to improve the accuracy of predicting flight trajectory in air traffic research, which is further applied to implement a probabilistic conflict detection algorithm. The Gaussian distribution is applied to serve as the probabilistic representation for illustrating the transition patterns of the flight trajectory, based on which a stochastic process is generated to build the temporal correlations among flight positions, i.e., Gaussian process (GP). Furthermore, to deal with the flight maneuverability of performing controller’s instructions, a hierarchical neural network architecture is proposed to improve the modeling representation for nonlinear features. Thanks to the intrinsic mechanism of the GP regression, the DGP model has the ability of predicting both the deterministic nominal flight trajectory (NFT) and its confidence interval (CI), denoting by the mean and standard deviation of the prediction sequence, respectively. The CI subjects to a Gaussian distribution, which lays the data foundation of the probabilistic conflict detection. Experimental results on real data show that the proposed trajectory prediction approach achieves higher prediction accuracy compared to other baselines. Moreover, the conflict detection approach is also validated by a obtaining lower false alarm and more prewarning time.

show abstract

Methods, progresses, and opportunities of materials informatics

Li,

Zheng

2023

InfoMat

View full text Add to dashboard Cite

As an implementation tool of data intensive scientific research methods, machine learning (ML) can effectively shorten the research and development (R&D) cycle of new materials by half or even more. ML shows great potential in the combination with other scientific research technologies, especially in the processing and classification of large amounts of material data from theoretical calculation and experimental characterization. It is very important to systematically understand the research ideas of material informatics to accelerate the exploration of new materials. Here, we provide a comprehensive introduction to the most commonly used ML modeling methods in material informatics with classic cases. Then, we review the latest progresses of prediction models, which focus on new processing–structure–properties–performance (PSPP) relationships in some popular material systems, such as perovskites, catalysts, alloys, two‐dimensional materials, and polymers. In addition, we summarize the recent pioneering researches in innovation of material research technology, such as inverse design, ML interatomic potentials, and microtopography characterization assistance, as new research directions of material informatics. Finally, we comprehensively provide the most significant challenges and outlooks related to the future innovation and development in the field of material informatics. This review provides a critical and concise appraisal for the applications of material informatics, and a systematic and coherent guidance for material scientists to choose modeling methods based on required materials and technologies.

show abstract

A Unified Framework for Multilingual Speech Recognition in Air Traffic Control Systems

Cited by 46 publications

References 46 publications

Towards multilingual end‐to‐end speech recognition for air traffic control

Towards multilingual end‐to‐end speech recognition for air traffic control

A Deep Gaussian Process-Based Flight Trajectory Prediction Approach and Its Application on Conflict Detection

Methods, progresses, and opportunities of materials informatics

Contact Info

Product

Resources

About