Light Stemming for Arabic Information Retrieval

Larkey, Leah S.; Ballesteros, Lisa; Connell, Margaret E.

doi:10.1007/978-1-4020-6046-5_12

Cited by 174 publications

(130 citation statements)

References 25 publications

Supporting

Mentioning

128

Contrasting

Order By: Relevance

“…This approach can be calculated as the number of doc-ument relevant and retrieve (tp) from total number of relevant documents in collections (tp + tn). The F-measured approach can be created by combining precision and recall as shown in equation (10).…”

Section: Results and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Term Weighting Based on Positive Impact Factor Query for Arabic Fiqh Document Ranking

Sholikah

Kartika

Arifin

et al. 2017

Jurnal Ilmu Komputer dan Informasi

View full text Add to dashboard Cite

Query becomes one of the most decisive factor on documents searching. A query contains several words, where one of them will become a key term. Key term is a word that has higher information and value than the others in query. It can be used in any kind of text documents, including Arabic Fiqh documents. Using key term in term weighting process could led to an improvement on result's relevancy. In Arabic Fiqh document searching, not using the proper method in term weighting will relieve important value of key term. In this paper, we propose a new term weighting method based on Positive Impact Factor Query (PIFQ) for Arabic Fiqh documents ranking. PIFQ calculated using key term's frequency on each category (mazhab) on Fiqh. The key term that frequently appear on a certain mazhab will get higher score on that mazhab, and vice versa. After PIFQ values are acquired, TF.IDF calculation will be done to each words. Then, PIFQ weight will be combine with the result from TF.IDF so that the new weight values for each words will be produced. Experimental result performed on a number of queries using 143 Arabic Fiqh documents show that the proposed method is better than traditional TF.IDF, with 77.9%, 83.1%, and 80.1% of precision, recall, and F-measure respectively. Keywords: Document Ranking, Arabic, Term Weighting, Query, PIFQ. AbstrakQuery menjadi salah satu faktor penentu dalam pencarian dokumen. Dalam sebuah query terdiri dari beberapa kata, dimana salah satunya menjadi key term. Key term adalah kata yang memiliki nilai informasi dan bobot lebih tinggi dibandingkan kata lain. Hal tersebut berlaku untuk semua jenis dokumen teks, termasuk dokumen fiqih berbahasa Arab. Penitik beratan pada key term dalam proses pembobotan kata memungkinkan terjadinya peningkatan relevansi pencarian. Di dalam pencarian dokumen fiqih berbahasa Arab, jika metode pembobotan kata yang digunakan tidak tepat, key term tidak akan memberikan pengaruh berarti. Oleh karena itu diusulkanlah sebuah metode pembobotan baru pada kata berbasis Positive Impact Factor Query (PIFQ) untuk perangkingan dokumen fiqih berbahasa arab. PIFQ dihitung menggunakan frekuensi kemunculan key term pada setiap kategori (mazhab) dalam fiqih. Semakin tinggi frekuensi key term tersebut pada suatu mazhab semakin tinggi pula nilainya pada mazhab tersebut, begitu pula sebaliknya. Setelah didapat nilai PIFQ, kemudian dilakukan perhitungan TF.IDF untuk setiap kata. Selanjutnya bobot PIFQ akan dikom-binasikan dengan TF.IDF sehingga menghasilkan bobot baru untuk masing-masing kata. Hasil dari pengujian yang dilakukan pada sejumlah query dengan 143 dokumen fiqih berbahasa Arab menunjukan bahwa metode usulan dapat lebih unggul jika dibandingkan metode TF.IDF, dengan nilai precision, recall, dan Fmeasure masing-masing sebesar 77,9%, 83,1%, dan 80,1%.

show abstract

Section: Results and Analysismentioning

confidence: 99%

“…In this study we used Light Stemmer [10]. Light stemmer is one of the method to find root in Arabic without using dictionary.…”

Section: Preprocessingmentioning

confidence: 99%

Term Weighting Based on Positive Impact Factor Query for Arabic Fiqh Document Ranking

Sholikah

Kartika

Arifin

et al. 2017

Jurnal Ilmu Komputer dan Informasi

View full text Add to dashboard Cite

show abstract

“…If not explicitly stated otherwise, the training set size for each category is 100 documents and version 3 of the dataset is used. This version represents the original documents modified by the removal of stop words and application of the light10 stemmer [11], considered by many researchers to be the best stemmer for the Arabic language. The accuracies presented in the graphs below are always the average of five independent runs, unless the accuracies of all five runs are plotted in the associated graph.…”

Section: Testing Methodology Experiments and Resultsmentioning

confidence: 99%

Arabic Document Classification Using Multiword Features

Abuaiadah¹

2013

IJCCE

View full text Add to dashboard Cite

Abstract-Weinvestigate the use of multiword features to improve Arabic document classification. The Arabic language is both morphologically rich and highly inflected. Accordingly it presents more challenges when enhancing Arabic information retrieval to a level comparable to English. The multiword features are modeled as a combination of words appearing within windows of varying sizes. Our experiments show multiword features combined with dice similarity distance outperform the cosine similarity function and produce results that are comparable to TF-IDF representation. Multiword features are under-explored and we believe they have the potential to improve Arabic information retrieval and, in particular, Arabic document classification.

show abstract

“…Arabic is a rich Semitic language which is highly productive, both derivationally and inflectionally [2,4]. The number of Arabic words is estimated to be 60 billion, derived from approximately 10,000 roots.…”

Section: An Event That Interferes the Achieving Of The Objective Of Amentioning

confidence: 99%

“…Most attention is focused on text classification, techniques used for language preprocessing like (stemmers and index tools), filtering and translation [2,4]. Previous work on Arabic IR has used distance-based algorithms, Learning algorithms, Bayesian classification methods and N-grams for searching Arabic text documents [4].…”

Section: Related Workmentioning

confidence: 99%

Arabic Event Detection in Social Media

Alsaedi

Burnap

2015

Computational Linguistics and Intelligent Text Processing

View full text Add to dashboard Cite

Abstract. Event detection is a concept that is crucial to the assurance of public safety surrounding real-world events. Decision makers use information from a range of terrestrial and online sources to help inform decisions that enable them to develop policies and react appropriately to events as they unfold. One such source of online information is social media. Twitter, as a form of social media, is a popular micro-blogging web application serving hundreds of millions of users. User-generated content can be utilized as a rich source of information to identify real-world events. In this paper, we present a novel detection framework for identifying such events, with a focus on 'disruptive' events using Twitter data. The approach is based on five steps; data collection, preprocessing, classification, clustering and summarization. We use a Naïve Bayes classification model and an Online Clustering method to validate our model over multiple real-world data sets. To the best of our knowledge, this study is the first effort to identify real-world events in Arabic from social media.

show abstract

Light Stemming for Arabic Information Retrieval

Cited by 174 publications

References 25 publications

Term Weighting Based on Positive Impact Factor Query for Arabic Fiqh Document Ranking

Term Weighting Based on Positive Impact Factor Query for Arabic Fiqh Document Ranking

Arabic Document Classification Using Multiword Features

Arabic Event Detection in Social Media

Contact Info

Product

Resources

About