On the use of machine translation-based approaches for vietnamese diacritic restoration

Abstract

This paper presents an empirical study of two machine translation-based approaches for Vietnamese diacritic restoration problem, including phrase-based and neural-based machine translation models. This is the first work that applies neural-based machine translation method to this problem and gives a thorough comparison to the phrase-based machine translation method which is the current state-of-the-art method for this problem. On a large dataset, the phrase-based approach has an accuracy of 97.32% while that of the neural-based approach is 96.15%. While the neural-based method has a slightly lower accuracy, it is about twice faster than the phrase-based method in terms of inference speed. Moreover, neural-based machine translation method has much room for future improvement such as incorporating pre-trained word embeddings and collecting more training data.

Efficient Convolutional Neural Networks for Diacritic Restoration

“…While these deep models achieve state-of-the-art performance, they mainly rely on the use of recurrent architectures such as BiLSTM, which are relatively inefficient. Pham et al (2017) view the task of diacritization for Vietnamese as a machine transduction problem from undiacritized to diacritized text at the word level. Orife (2018) addresses the problem on Yoruba in a similar way and compares softand self-attention sequence-to-sequence performance on the word level empirically showing that self-attention significantly outperforms BiLSTM.…”

“…Feature engineering and classical machine learning algorithms such as Hidden Markov Models, Maximum Entropy Models, and Finite State Transducer were the dominant approaches (Nelken and Shieber, 2005;Zitouni et al, 2006;Elshafei et al, 2006). However, recent studies show significant improvement using deep neural networks (Belinkov and Glass, 2015;Pham et al, 2017;Orife, 2018). While these deep models achieve state-of-the-art performance, they mainly rely on the use of recurrent architectures such as BiLSTM, which are relatively inefficient.…”

Statistical machine translation outperforms neural machine translation in software engineering: why and how

“…In SE researches, the problem of Type Inference using MT shows that SMT model provided by [37] has a significant higher accuracy compared to the original NMT approach in [18]. Similarly, for natural language diacritic restoration, [35] shows that SMT outperforms NMT. [18,30,37] have the same characteristics of parallel corpus, i.e., the length of source and target pairs are equal and the order of the source and target words are consistent with each other.…”

“…2 PREFIX RESOLUTION [18,37] provide a translation approach considered the source side language as partial class name (PCN) and the target language as Fully Qualified Name (FQN) of APIs. [35] treated the source language as a word without diacritic information and the target language as a word with diacritic information. In other words, both of these research works build a parallel corpus with the same length of source sequence and target sequence to each pair.…”

“…The orders for source and for target sequences are also consistent. In summary, the problem of translation in [18,35,37] has the common in characteristics of parallel corpus. In our work, we design an inferring system based on translation inspiring from [37].…”