Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation

Abstract

This paper proposes a sophisticated neural architecture to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models. By introducing three novel components: Pointer, Disambiguator, and Copier, our method PDC achieves the following merits inherently compared with previous efforts:(1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionaries can potentially be used;(2) Disambiguator synthesizes contextual information from the source view and the target view, both of which contribute to distinguishing the proper translation of a specific source word from multiple candidates in dictionaries;(3) Copier systematically connects Pointer and Disambiguator based on a hierarchical copy mechanism seamlessly integrated with Transformer, thereby building an end-to-end architecture that could avoid error propagation problems in alternative pipeline methods. The experimental results on Chinese-English and English-Japanese benchmarks demonstrate the PDC's overall superiority and effectiveness of each component.

DPNPED: Dynamic Perception Network for Polysemous Event Trigger Detection

“…The initial embedding is based on BERT and the hyperparameter is the same as their paper. GatedGCN exploits gate diversity and the overall contextual importance scores of the words on graph convolution neural networks 4 . The dependency tree is built on Stanford CoreNLP toolkit.…”

“…MOGANED code is available in https://github.com/ll0ruc/MOGANED3 DMBERT code is available in https://github.com/Bakser/DMBERT4 GatedGCN code is available in https://github.com/laiviet/ed-gated-gcn 5 EE-GCN code is available in https://github.com/cuishiyao96/eegcned6 MLBiNet code is available in https://github.com/zjunlp/DocED…”

Frequency-Aware Contrastive Learning for Neural Machine Translation

“…Low-Frequency Word Translation is a persisting challenge for NMT due to the token imbalance phenomenon. Conventional researches range from introducing finegrained translation units (Luong and Manning 2016;Lee, Cho, and Hofmann 2017), seeking optimal vocabulary (Wu et al 2016;Sennrich, Haddow, and Birch 2016;Gowda and May 2020;Liu et al 2021), to incorporating external lexical knowledge (Luong et al 2015;Arthur, Neubig, and Nakamura 2016;Zhang et al 2021). Recently, some approaches alleviate this problem by well-designed loss function with adaptive weights, in light of the token frequency (Gu et al 2020) or bilingual mutual information (Xu et al 2021b).…”

“…Implementation Details We examine our model based on the advanced Transformer architecture and base setting (Vaswani et al 2017). All the baseline systems and our models are implemented on top of THUMT toolkit (Zhang et al 2017). During training, the dropout rate and label smoothing are set to 0.1.…”