Automatic Word Spacing of Korean Using Syllable and Morpheme

Abstract

In Korean, spacing is very important to understand the readability and context of sentences. In addition, in the case of natural language processing for Korean, if a sentence with an incorrect spacing is used, the structure of the sentence is changed, which affects performance. In the previous study, spacing errors were corrected using n-gram based statistical methods and morphological analyzers, and recently many studies using deep learning have been conducted. In this study, we try to solve the spacing error correction problem using both the syllable-level and morpheme-level. The proposed model uses a structure that combines the convolutional neural network layer that can learn syllable and morphological pattern information in sentences and the bidirectional long short-term memory layer that can learn forward and backward sequence information. When evaluating the performance of the proposed model, the accuracy was evaluated at the syllable-level, and also precision, recall, and f1 score were evaluated at the word-level. As a result of the experiment, it was confirmed that performance was improved from the previous study.

€œAPA NAMANYA”: KAJIAN SENYAPAN FRASA PENJEDA PADA PEMBICARAAN PUBLIK

“…Penjedaan tersebut jika dilakukan dengan tidak tepat bahkan sebagai sebuah ketidaksengajaan yang merupakan akibat dari faktor internal dari pembicara, dapat menimbulkan makna yang tidak sesuai sehingga gagasan pembicara tidak tersampaikan. Dalam hal ini, penjedaan sebuah ujaran sangat berpengaruh terhadap makna yang ditimbulkan (Choi et al, 2021).…”

BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text

“…Spacing The first limitation is related to segmentation, i.e., the spaces are generally not adequately separated in the speech recognition result. To solve this problem, many studies have investigated an automatic spacing module; however, few studies have focused on ASR (Lee and Kim, 2013;Choi et al, 2021). Thus, the satisfactoriness of ASR service, which is used by end users, is low, and the speech recognition results will lack credibility if this problem is not resolved.…”