Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Abstract

We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no changes to the model architecture from a standard NMT system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes an encoder, decoder and attention module, remains unchanged and is shared across all languages. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT using a single model without any increase in parameters, which is significantly simpler than previous proposals for Multilingual NMT. On the WMT'14 benchmarks, a single multilingual model achieves comparable performance for English→French and surpasses state-of-the-art results for English→German. Similarly, a single multilingual model surpasses state-of-the-art results for French→English and German→English on WMT'14 and WMT'15 benchmarks, respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. In addition to improving the translation quality of language pairs that the model was trained with, our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and show some interesting examples when mixing languages.

Latent Space Cartography: Visual Analysis of Vector Space Embeddings

“…Researchers have also developed interactive visual analysis tools for latent spaces [STN∗16, JSL∗17, HG18, LBT∗18, LNH∗18]. Some tools focus on a subset of tasks [STN∗16, LBT∗18] in word embeddings, which we extend and bring to a broader range of latent spaces.…”

Enterprise data breach: causes, challenges, prevention, and future directions

“…[57][58][59] Deep learning techniques have recently shown promises in solving complex sequence-to-sequence translation problems in natural languages. 60 Training deep learners to infer sequences of user intentions based on sequences of machine events is an extremely interesting direction. • DLPD as a Cloud Service: The advent of cloud computing offers a new option for conducting data leak detection.…”

Consistency by Agreement in Zero-Shot Neural Machine Translation

“…Luong et al (2015a); Firat et al (2016a) proposed to use O(k) encoders/decoders that are then intermixed to translate between language pairs. Johnson et al (2016) proposed to use a single model and prepend special symbols to the source text to indicate the target language, which has later been extended to other text preprocessing approaches (Ha et al, 2017) as well as languageconditional parameter generation for encoders and decoders of a single model (Platanios et al, 2018). Johnson et al (2016) also show that a single multilingual system could potentially enable zeroshot translation, i.e., it can translate between language pairs not seen in training.…”