Streaming End-to-End Bilingual ASR Systems with Joint Language Identification

Punjabi, Surabhi; Arsikere, Harish; Raeesy, Zeynab; Chandak, Chander; Bhave, Nikhil Aniruddha; Bansal, Ankish; Müller, Markus; Murillo, Sergio; Rastrow, Ariya; Garimella, Sri; Maas, Roland; Hans, Mat; Athanasios, Mouchtaris,; Kunzmann, Siegfried

doi:10.48550/arxiv.2007.03900

Cited by 10 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This could be the reason why the confusion rates are similar. To further reduce the confusion rate, we may consider various language identification approaches [1], [2], [18]. We calculate the average length of hypotheses from the bilingual experiments.…”

Section: Language Confusionmentioning

confidence: 99%

Bilingual End-to-End ASR with Byte-Level Subwords

Deng¹,

Hsiao²,

Ghoshal³

2022

Preprint

View full text Add to dashboard Cite

In this paper, we investigate how the output representation of an end-to-end neural network affects multilingual automatic speech recognition (ASR). We study different representations including character-level, byte-level, byte pair encoding (BPE), and bytelevel byte pair encoding (BBPE) representations, and analyze their strengths and weaknesses. We focus on developing a single end-toend model to support utterance-based bilingual ASR, where speakers do not alternate between two languages in a single utterance but may change languages across utterances. We conduct our experiments on English and Mandarin dictation tasks, and we find that BBPE with penalty schemes can improve utterance-based bilingual ASR performance by 2% to 5% relative even with smaller number of outputs and fewer parameters. We conclude with analysis that indicates directions for further improving multilingual ASR.

show abstract

Section: Language Confusionmentioning

confidence: 99%

Bilingual End-to-End ASR with Byte-Level Subwords

Deng¹,

Hsiao²,

Ghoshal³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The work developed in [15] develops an LID model that uses acoustic and text embeddings to choose the correct ASR model in 4 different languages. The work in [16] uses pre-trained LID embeddings to choose between ASR models in English-Spanish and English-Hindi pairs.…”

Section: Related Workmentioning

confidence: 99%

Bilingual Asr Model With Language Identification for Brazilian Portuguese and South-American Spanish

Farias¹,

Lobato²,

Castañeda³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper documents the development of a special case of multilingual Automatic Speech Recognition model, specifically tailored to attend two languages spoken by the majority of Latin America, Portuguese and Spanish. The bilingual model combines Language Identification and Speech Recognition developed with the Wav2Vec2.0 architecture and trained on several open and private speech datasets. In this model, the feature encoder is trained jointly for all tasks and different context encoders are trained for each task. The model is evaluated separately on two tasks: language identification and speech recognition. The results indicate that this model achieves good performance on speech recognition and average performance on language identification, training on a low quantity of speech material. The average accuracy of the language identification module on the MLS dataset is 66.75%. The average Word Error Rate in the same scenario is 13.89%, which is better than average 22.58% achieved by the commercial speech recognizer developed by Google.

show abstract

“…To remove the dependency on knowing one-hot LID in advance, one way is to estimate the LID and use it as the additional input to E2E multilingual models. However, the gain is very limited, especially for streaming E2E models, because the estimation is not very reliable [138,139]. Another solution is to build a corresponding multilingual E2E model for any set of language combinations.…”

Section: Multilingual Modelingmentioning

confidence: 99%

Recent Advances in End-to-End Automatic Speech Recognition

Li¹

2021

Preprint

View full text Add to dashboard Cite

Recently, the speech community is seeing a significant trend of moving from deep neural network based hybrid modeling to end-to-end (E2E) modeling for automatic speech recognition (ASR). While E2E models achieve the state-of-the-art results in most benchmarks in terms of ASR accuracy, hybrid models are still used in a large proportion of commercial ASR systems at the current time. There are lots of practical factors that affect the production model deployment decision. Traditional hybrid models, being optimized for production for decades, are usually good at these factors. Without providing excellent solutions to all these factors, it is hard for E2E models to be widely commercialized. In this paper, we will overview the recent advances in E2E models, focusing on technologies addressing those challenges from the industry's perspective.

show abstract

Streaming End-to-End Bilingual ASR Systems with Joint Language Identification

Cited by 10 publications

References 28 publications

Bilingual End-to-End ASR with Byte-Level Subwords

Bilingual End-to-End ASR with Byte-Level Subwords

Bilingual Asr Model With Language Identification for Brazilian Portuguese and South-American Spanish

Recent Advances in End-to-End Automatic Speech Recognition

Contact Info

Product

Resources

About