A comparative study on Thai word segmentation approaches

Haruechaiyasak, Choochart; Kongyoung, Sarawoot; Dailey, Matthew N.

doi:10.1109/ecticon.2008.4600388

Cited by 80 publications

(34 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In indexed Thai texts using an inverted index [5], word segmentation [6] - [10] is one of the most widely used information extraction techniques in Natural Language Processing (NLP). The word segmentation technique is used to perform the index terms tokenization.…”

Section: Thai Index Terms Extractionmentioning

confidence: 99%

See 1 more Smart Citation

An automatic indexing technique for Thai texts using frequent max substring

Chumwatana

Wong

Xie

2009

2009 Eighth International Symposium on Natural Language Processing

View full text Add to dashboard Cite

Abstract-Thai language is considered as a non-segmented language where words are a string of symbols without explicit word boundaries, and also the structure of written Thai language is highly ambiguous. This problem causes an indexing technique has become a main issue in Thai text retrieval. To construct an inverted index for Thai texts, an index terms extraction technique is usually required to segment texts into index term schemes. Although index terms can be specified manually by experts, this process is very time consuming and labor-intensive. Word segmentation is one of the many techniques that are used to automatically extract index terms from Thai texts. However, most of the word segmentation techniques require linguistic knowledge and the preparation of these approaches is time consuming. An ngram based approach is another automatic index terms extraction method that is often used as indexing technique for Asian languages including Thai. This approach is language independent which does not require any linguistic knowledge or dictionary. Although the n-gram approach out performs many indexing techniques for Asian languages in term of retrieval effectiveness, the disadvantage of n-gram approach is it suffers from large storage space and long retrieval time. In this paper we present the frequent max substring mining to extract index terms from Thai texts. Our method is languageindependent and it does not rely on any dictionary or language grammatical knowledge. Frequent max substring mining is based on text mining that describes a process of discovering useful information or knowledge from unstructured texts. This approach uses the analysis of frequent max substring sets to extract all long and frequently-occurred substrings. We aim to employ the frequent max substring mining algorithm to address the drawback of n-gram based approach by keeping only frequent max substrings to reduce disk space requirement for storing index terms and to reduce the retrieval time in order to deal with the rapid growth of Thai texts.

show abstract

Section: Thai Index Terms Extractionmentioning

confidence: 99%

“…After which, these segmented index terms will be stored into the inverted index structure. Most techniques are based on word segmentation which usually relies on any dictionary or requires the linguistic knowledge of the language [3], [6]. However, there is some other techniques which do not rely on language analysis.…”

Section: Introductionmentioning

confidence: 99%

An automatic indexing technique for Thai texts using frequent max substring

Chumwatana

Wong

Xie

2009

2009 Eighth International Symposium on Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…This research interested in THAI language. The best algorithm for THAI segmentation is Longest Word Matching Algorithm (7) as shown in Figure 2. Fig.…”

Section: 12mentioning

confidence: 99%

Automatic Advertisement System using Text Mining Technique

Jareanpon

2015

Proceedings of the 3rd International Conference on Intelligent Systems and Image Processing 2015

View full text Add to dashboard Cite

Nowadays, there are a wide variety of advertising types. Indeed, one of them is the Internet advertising and it is growing rapidly. However, with the different behaviors of consumers who visit a web site, many websites are developed and proposed in order to meet the needs of many different people. Indeed, if a website is interested by many people, rating of the website is increased. However, after accessing to the website, if the Internet advertisings found on the website does not relate with the consumer interests, these online advertising may not be effective. As the result, this work presents a method of text mining approach to suggest the online advertisements that are relevant to the content of the website. After experimenting by the Random Forest and Bayesian Network classifier, the result of Random Forest is better than the result of Naïve Bayes, where the Random Forest shows the accuracy at 83.97%

show abstract

“…In the automated process, unknown word boundaries are identified using frequencies of strings. In [20], a comparison of dictionary-based approach and ML-based approach for word segmentation was presented where unknown word detection is implicitly handled. Since either of the dictionarybased and ML-based approaches has its advantages, most previous works [2], [5], [15] combined them to handle unknown words.…”

Section: Previous Workmentioning

confidence: 99%

A Corpus-Based Approach for Automatic Thai Unknown Word Recognition Using Boosting Techniques

TeCho

Nattee

Theeramunkong

2009

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Jakkrit TECHO†a) , Student Member, Cholwich NATTEE †b) , and Thanaruk THEERAMUNKONG †c) , Members SUMMARY While classification techniques can be applied for automatic unknown word recognition in a language without word boundary, it faces with the problem of unbalanced datasets where the number of positive unknown word candidates is dominantly smaller than that of negative candidates. To solve this problem, this paper presents a corpus-based approach that introduces a so-called group-based ranking evaluation technique into ensemble learning in order to generate a sequence of classification models that later collaborate to select the most probable unknown word from multiple candidates. Given a classification model, the group-based ranking evaluation (GRE) is applied to construct a training dataset for learning the succeeding model, by weighing each of its candidates according to their ranks and correctness when the candidates of an unknown word are considered as one group. A number of experiments have been conducted on a large Thai medical text to evaluate performance of the proposed group-based ranking evaluation approach, namely V-GRE, compared to the conventional naïve Bayes classifier and our vanilla version without ensemble learning. As the result, the proposed method achieves an accuracy of 90.93±0.50% when the first rank is selected while it gains 97.26±0.26% when the top-ten candidates are considered, that is 8.45% and 6.79% improvement over the conventional record-based naïve Bayes classifier and the vanilla version. Another result on applying only best features show 93.93±0.22% and up to 98.85 ±0.15% accuracy for top-1 and top-10, respectively. They are 3.97% and 9.78% improvement over naive Bayes and the vanilla version. Finally, an error analysis is given. key words: unknown word recognition, word boundary detection, data mining, machine learning, ensemble learning

show abstract

A comparative study on Thai word segmentation approaches

Cited by 80 publications

References 3 publications

An automatic indexing technique for Thai texts using frequent max substring

An automatic indexing technique for Thai texts using frequent max substring

Automatic Advertisement System using Text Mining Technique

A Corpus-Based Approach for Automatic Thai Unknown Word Recognition Using Boosting Techniques

Contact Info

Product

Resources

About