Replacing Out-of-Vocabulary Words with an Appropriate Synonym Based on Word2VnCR

Abstract

The most typical problem in an analysis of natural language is finding synonyms of out-of-vocabulary (OOV) words. When someone tries to understand a sentence containing an OOV word, the person determines the most appropriate meaning of a replacement word using the meanings of co-occurrence words under the same context based on the conceptual system learned. In this study, a word-to-vector and conceptual relationship (Word2VnCR) algorithm is proposed that replaces an OOV word leading to an erroneous morphemic analysis with an appropriate synonym. TheWord2VnCR algorithm is an improvement over the conventional Word2Vec algorithm, which has a problem in suggesting a replacement word by not determining the similarity of the word. After word-embedding learning is conducted using the learning dataset, the replacement word candidates of the OOV word are extracted. The semantic similarities of the extracted replacement word candidates are measured with the surrounding neighboring words of the OOV word, and a replacement word having the highest similarity value is selected as a replacement. To evaluate the performance of the proposed Word2VnCR algorithm, a comparative experiment was conducted using the Word2VnCR and Word2Vec algorithms. As the experimental results indicate, the proposed algorithm shows a higher accuracy than the Word2Vec algorithm.

Handling Compound Hindi OOV words in web queries

“…Jeongin Kim et al (2021) [21] introduced a Word2VnCR algorithm to replace an OOV word with a semantically related term when an error occurs in morpheme analysis. With the help of this approach, candidate words to be exchanged with the OOV word having the same meaning as OOV are extracted and their semantic similarity to the OOV word's nearby terms can be determined.…”

Multilabel Emotion Recognition Technique considering the Characteristics of Unstructured Conversation Data

“…However, in the classification of the supervised learning method using dictionary or vocabulary features, the problem of insufficient data is the most problematic. Various emotion classes and vocabulary used for each class appear in various ways [20]. Therefore, it is very difficult to construct sufficient learning data to enable learning at an appropriate level using these vocabularies.…”