A New Approach To Accent Restoration Of Vietnamese Texts Using Dynamic Programming Combined With Co-Occurrence Graph

Abstract

In this paper, we would like to introduce a new approach to recover Vietnamese text's accents. Given a Vietnamese text in which accents are lost, our goal is to seek for a recovered text that yields a best lexical probability. Using a dynamic programming approach, we first build a model of language for Vietnamese as a lexical database which gives lexical probabilities to Vietnamese sentences. Second, we construct a map of literal translations of Vietnamese words to restrict our searching space. Finally, we apply dynamic programming as a searching engine to seek out the most probable sentence. We also use the co-occurrence graph to increase the accuracy of selection. the experimental results show that the average accuracy of our approach is about 93% -94%.

Vietnamese Diacritics Restoration as Sequential Tagging

“…Some classification methods include Memory-Based Learning [4], Hidden Markov Models [5]. In Vietnamese, to our knowledge, there are some researches [6], [7], [8] and some software packages for restoring accents, such as AMPad 1 and VietPad 2 .…”

“…An interesting approach was proposed by Nghia et al [7], in which, they combined both language model and co-occurrence graph to capture the context information within non-accent texts, and then they applied dynamic programming to seek for the original texts that best satisfies the collected context information. For more details, they used the local information (the context within a sentence) to reconstruct an original sentence which is soundness and they also integrated the global information (the content within a whole text -the domain) to avoid ambiguity in semantic meanings.…”

“…Maybe, with good feature selection, it will give a better performance than CRFs in this task.It is difficult to compare our results with other results in[6],[7],[8] because the training data and test data are different. Nevertheless, the experimental results (93.81 % in the best case) show that our methods are competitive with other methods for solving this problem.…”