Enhancing Malay Stemming Algorithm with Background Knowledge

Leong, Leow Ching; Basri, Surayaini Binti; Alfred, Rayner

doi:10.1007/978-3-642-32695-0_68

Cited by 18 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to do that, we may need a lexicon Malay dictionary that consists of Malay word and its type of category [4]. Besides that, we may add some rules in order to detect and correct the affixed English word, the loan word and also to improve the stemming process as already discussed in [3].…”

Section: Discussionsupporting

confidence: 45%

Automatic spell checker for Malay blog

Basri

Alfred

2012

2012 IEEE International Conference on Control System, Computing and Engineering

Self Cite

View full text Add to dashboard Cite

Spell checker is a system that is used to detect and correct misspelled word. Misspelled word is a word that exists in the existing lexicon that is not correctly spelled or in shortened form. These misspelled words often result in ineffective results of the Information Retrieval (IR) application such as document retrieval. This is because IR application should be able to recognize all words in a particular language in order to be more robust. The current spell checker for the Malay language uses a dictionary that contains pair of commonly misspelled word and its correctly spelled word in detecting and correcting misspelled word. However, this type of spell checker can only correct misspelled words that exist in the existing dictionary; otherwise it requires user interaction to correct it manually. This approach works well if the spell checker is a standalone system but it is not really an effective system when the spell checker is part of another IR application such as document retrieval for weblog. This is because there will be always new misspelled words created along with the increasing number of weblog pages. Thus, the number of misspelled words will also grow extremely. In this paper, we propose a new spell checker that detects and automatically corrects misspelled words in Malay without any interaction from the user. The proposed approach automatically replaces the misspelled word if it exists in the reSpellWord dictionary. Otherwise, it will go through the process of Selangor Slang Identification or Repetitive word Identification or Opposite Word Identification whichever is suitable. If the word cannot be identified as a misspelled word, a few alternative words will be suggested and they are ranked using the Levenshtein Distance in order to choose the most likelihood word for the misspelled word. The correctly-spelled word that has the highest ranking will be chosen as a replacement for the misspelled word. This misspelled word and its correctly-spelled word are then added automatically into the dictionary in order to update the dictionary. The proposed approach is evaluated by using texts that are selected randomly from the popular Malay blog. Based on the experimental results obtained, the proposed approach is found to be effective in detecting and correcting the Malay misspelled word automatically.Index Terms-Malay weblog, information retrieval, misspelled word, spell checker

show abstract

Section: Discussionsupporting

confidence: 45%

Automatic spell checker for Malay blog

Basri

Alfred

2012

2012 IEEE International Conference on Control System, Computing and Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…For English documents we are using the Porter Stemming Algorithm [41]. As for Malay documents, we are using the stemmer that developed in [42]. The result of the tf.idf calculation will be store in a data document for further analysis and reviewed.…”

Section: F Document Statisticsupporting

confidence: 42%

A Robust Framework for Web Information Extraction and Retrieval

Alfred¹,

Soon²,

On³

et al. 2014

IJMLC

Self Cite

View full text Add to dashboard Cite

Abstract-The large volume of online and offline information that is available today has overwhelmed users' efficiency and effectiveness in processing this information in order to extract relevant information. The exponential growth of the volume of Internet information complicates information access. Thus, it is a very time consuming and complex task for user in accessing relevant information. Information retrieval (IR) is a branch of artificial intelligence that tackles the problem of accessing and retrieving relevant information. The aim of IR is to enable the available data source to be queried for relevant information efficiently and effectively. This paper describes a robust information retrieval framework that can be used to retrieve relevant information. The proposed information retrieval framework is designed to assist users in accessing relevant information effectively and efficiently as it handles queries based on user preferences. Each component and module involved in the proposed framework will be explained in terms of functionality and the processes involved.Index Terms-Information retrieval, information retrieval framework, semantic web. I. INTRODUCTIONInformation retrieval (IR) is a process that extracts and retrieves information that is relevant to user based on the queries posted. IR deals with many aspects including the representation, storage, organization and retrieving information from data sources. Furthermore, these data sources can be accessed offline or online and they can be categorized into structured, semi-structured or unstructured data. The origin of the IR research can be traced back to ancient times when librarians kept information related to articles or books using catalogue cards [1], [2] and earlier works related to information retrieval can be found in 1950 [3]. The advent of computer has brought the IR system to a new level as computers are capable of processing large volume of data in order to extract and retrieve relevant information [4]. The increase of capacity and computational power has contributed to the rapid growth of unstructured data. For instance, with the advent of World Wide Web(WWW) making the information available online through hyperlink, the research attention of IR have been Rayner Alfred, Gan Kim Soon, and Chin Kim On are with the COESA, Universiti Malaysia Sabah, 88400 Kota Kinabalu, Sabah, g_k_s967@yahoo.com, kimonchin@ums.edu.my).Patricia Anthony is with the Department of Applied Computing, Faculty of Environment, Society and Design, Lincoln University, Christchurch, New Zealand (e-mail: patricia.anthony@lincoln.ac.nz).shifted to Web IR and it is increasingly gaining popularity. Among significant IR tools for WWW IR are the search engines. In order to retrieve information from the WWW, search engines with different capabilities and algorithm shave been developed. However, the advancement of Internet made information available growth exponential through time and a robust framework for web information extraction and retrieval is critically required to pro...

show abstract

“…Kajian-kajian lepas yang berkaitan cantasan perkataan dalam Bahasa Melayu kebanyakannya hanya memfokuskan cantasan untuk perkataan Rumi sahaja. Contoh kajian tersebut adalah seperti kajian , Idris (2001), Leong et al (2012), Muhamad Taufik et al (2009) dan Fadzli et al (2012.…”

Section: Kajian-kajian Lepasunclassified

Pencantas Perkataan Jawi Lama (Old Jawi Stemmer) dalam Bahasa Melayu berasaskan Petua

C.W.Ahmad,

Omar,

Nasruddin

et al. 2021

MyJICT

View full text Add to dashboard Cite

Pencantas perkataan berfungsi untuk membuang imbuhan sesuatu perkataan dengan menghasilkan kata dasar bagi perkataan tersebut. Cantasan banyak digunakan dalam bidang pemprosesan bahasa tabii(PBT) seperti transliterasi mesin, penterjemahan mesin dan capaian dokumen. Dengan penggunaan cantasan, saiz kamus dapat dikurangkan kerana perkataan dalam morfologi yang sama tidak perlu dimasukkan berulang kali. Sebaliknya perkataan-perkataan tersebut dimasukkan dalam kumpulan yang sama. Dalam Bahasa Melayu, terdapat dua jenis skrip penulisan sama ada menggunakan iaitu sistem ejaan Rumi atau sistem ejaan Jawi. Banyak kajian berkaitan pencantas perkataan Bahasa Melayu lebih tertumpu kepada sistem ejaan Rumi berbanding dengan sistem ejaan Jawi. Kertas ini mencadangkan pencantas perkataan Melayu bagi aksara Jawi dengan menggunakan satu set peraturan dalam Jawi lama (satu set peraturan yang digunakan untuk mengekang pelbagai bentuk perkataan terbitan Jawi lama). Terdapat 187 petua yang digunakan untuk pencantas Jawi ini yang dipanggil sebagai PEJAL. Terdapat 2500 perkataan Jawi terbitan terdiri daripada awalan, apitan, akhiran, sisipan dan diuji menggunakan pencantas ini. Hasil uji kaji menunjukkan 88.5% daripada perkataan Jawi berjaya dicantas dengan betul. The word stemming works to remove the affix of a word by generating the base word for that word. Stemming is widely used in natural language processing (NLP) such as machine transliteration, machine translation and document access. With the use of stemming, the dictionary size can be reduced because words in the same morphology do not need to be entered repeatedly. Instead the words are included in the same group. There are two types of script writing in Malay language, either use the Latin spelling system or Jawi spelling system. Many studies on the Malay word grapple more focused on Latin spelling system compared with Jawi spelling system. This paper proposes Malay stemmer for old Jawi characters by using a set of rules in old Jawi (a set of rules used to constrain various forms of words derived from old Jawi). There are 187 rules was developed for the stemmer, called as PEJAL. There are 2500 derived Jawi words consisting of prefixes, suffixes, suffixes, insertions and tested using this stemmer. The experimental results showed that 88.5% of the Jawi words were successfully stemmed correctly.

show abstract

Enhancing Malay Stemming Algorithm with Background Knowledge

Cited by 18 publications

References 5 publications

Automatic spell checker for Malay blog

Automatic spell checker for Malay blog

A Robust Framework for Web Information Extraction and Retrieval

Pencantas Perkataan Jawi Lama (Old Jawi Stemmer) dalam Bahasa Melayu berasaskan Petua

Contact Info

Product

Resources

About