To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding?

Abstract

This paper addresses the question as to what degree a BERT-based multilingual Spoken Language Understanding (SLU) model can transfer knowledge across languages. Through experiments we will show that, although it works substantially well even on distant language groups, there is still a gap to the ideal multilingual performance. In addition, we propose a novel BERT-based adversarial model architecture to learn language-shared and language-specific representations for multilingual SLU. Our experimental results prove that the proposed model is capable of narrowing the gap to the ideal multilingual performance.

Upon Improving the Performance of Localized Healthcare Virtual Assistants

“…The majority of those low-resource languages lack of datasets or models and have usually not been thoroughly evaluated [ 12 ]. Moreover, the use of multilingual models cannot yet ensure similar performance with monolingual ones [ 13 ], as each language has its own unique characteristics that should be considered. Consequently, we cannot use pipelines implemented in English as-is for similar reasons.…”