Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Johnson, Melvin; Schuster, Mike; Le, Quoc V.; Krikun, Maxim; Wu, Yangjie; Chen, Zhifeng; Thorat, Nikhil; Viégas, Fernanda B.; Wattenberg, Martin; Corrado, Greg S.; Hughes, Macduff; Dean, Jeffrey

doi:10.48550/arxiv.1611.04558

Cited by 73 publications

(146 citation statements)

References 0 publications

Supporting

Mentioning

144

Contrasting

Order By: Relevance

“…2 The obvious difficulty in creating such an interface is data scarcity in the languages in question. In order to overcome these barriers, we plan to take advantage of recent advances in NLP that allow for multilingual modeling (Täckström et al, 2012;Johnson et al, 2016) and multi-task learning (Caruana, 1997), which allow models to be trained with very little, or even no data in the target language (Neubig and Hu, 2018). We also plan to utilize active learning (Settles, 2009), which specifically asks the linguists to focus on particular examples to maximize the effect of linguists' limited time when working with field data.…”

Section: Overall Frameworkmentioning

confidence: 99%

Towards a General-Purpose Linguistic Annotation Backend

et al. 2019

View full text Add to dashboard Cite

Language documentation is inherently a time-intensive process; transcription, glossing, and corpus management consume a significant portion of documentary linguists’ work. Advances in natural language processing can help to accelerate this work, using the linguists’ past decisions as training material, but questions remain about how to prioritize human involvement. In this extended abstract, we describe the beginnings of a new project that will attempt to ease this language documentation process through the use of natural language processing (NLP) technology. It is based on (1) methods to adapt NLP tools to new languages, based on recent advances in massively multilingual neural networks, and (2) backend APIs and interfaces that allow linguists to upload their data (§2). We then describe our current progress on two fronts: automatic phoneme transcription, and glossing (§3). Finally, we briefly describe our future directions (§4).

show abstract

Section: Overall Frameworkmentioning

confidence: 99%

Towards a General-Purpose Linguistic Annotation Backend

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Attention describes the tendency of visual processing to be confined largely to stimuli that are relevant to behavior (addressing the data efficiency). This topic has become an active research in image capationing (Xu et al, 2015), image generation (Gregor et al, 2015), VQA (Xiong et al, 2016), machine translation Johnson et al, 2016b), andspeech recognition (Chorowski et al, 2015). Specifically, Gregor et al (2015) began the early work in small sample learning with the deep recurrent attentive writer (DRAW) neural network architecture for image generation, where attention helped the system to build up an image incrementally, attending to one portion of a "mental canvas" at a time.…”

Section: Knowledge-driven Small Sample Learningmentioning

confidence: 99%

Small Sample Learning in Big Data Era

Shu,

Xu,

Meng

2018

Preprint

View full text Add to dashboard Cite

As a promising area in artificial intelligence, a new learning paradigm, called Small Sample Learning (SSL), has been attracting prominent research attention in the recent years. In this paper, we aim to present a survey to comprehensively introduce the current techniques proposed on this topic. Specifically, current SSL techniques can be mainly divided into two categories. The first category of SSL approaches can be called "concept learning", which emphasizes learning new concepts from only few related observations. The purpose is mainly to simulate human learning behaviors like recognition, generation, imagination, synthesis and analysis. The second category is called "experience learning", which usually co-exists with the large sample learning manner of conventional machine learning. This category mainly focuses on learning with insufficient samples, and can also be called small data learning in some literatures. More extensive surveys on both categories of SSL techniques are introduced and some neuroscience evidences are provided to clarify the rationality of the entire SSL regime, and the relationship with human learning process. Some discussions on the main challenges and possible future research directions along this line are also presented.

show abstract

“…The only difference from the standard encoderdecoder architecture with an attention mechanism (Bahdanau et al, 2015) is that in encoding, we concatenate u i−1 and u i−2 , and attach a i to the top of the long sentence as a special word. The technique here is similar to that in zero-shot machine translation (Johnson et al, 2016). Formulation details are given in Appendix.…”

Section: Supervised Learningmentioning

confidence: 99%

Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts

Xu,

Wu,

2018

Preprint

View full text Add to dashboard Cite

We study open domain dialogue generation with dialogue acts designed to explain how people engage in social chat. To imitate human behavior, we propose managing the flow of human-machine interactions with the dialogue acts as policies. The policies and response generation are jointly learned from humanhuman conversations, and the former is further optimized with a reinforcement learning approach. With the dialogue acts, we achieve significant improvement over state-of-the-art methods on response quality for given contexts and dialogue length in both machine-machine simulation and human-machine conversation.

show abstract

Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Cited by 73 publications

References 0 publications

Towards a General-Purpose Linguistic Annotation Backend

Towards a General-Purpose Linguistic Annotation Backend

Small Sample Learning in Big Data Era

Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts

Contact Info

Product

Resources

About