POS weighted TF-IDF algorithm and its application for an MOOC search engine

Xu, Ruilin

doi:10.1109/icalip.2014.7009919

Cited by 17 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…where c(t,d) indicates the occurrence of term t appears in document d, and the denominator P i cðt i ; dÞ indicates the total number of terms in document d, D is the total number of documents in the dataset, and d t is the number of the documents a term t appeared in. Many researchers [26][27][28]36] tries to improve the performance of TF-IDF by proposing some modification to the original equation of TF-IDF. In the case of Bloom's taxonomy verb plays an important role to determine the level of the question.…”

Section: Modified Tf-idf (Term Weighting Methods Tfpos-idf)mentioning

confidence: 99%

“…It assigns scores to the importance of a word in a document, based on the lexical and morphological properties of the text. It is extensively used in many studies [25][26][27][28][29], and in question classification [3,6,20,23,[30][31][32] However, some researchers used TF-IDF as it is, while others proposed some enhancement to the way TF-IDF is calculated in order to improve it. This is because classical TF-IDF does not capture some useful information such as the impact of word distribution between various classes [6].…”

Section: Related Workmentioning

confidence: 99%

“…A Massive Open Online Courses (MOOC) search engine proposed by [26], used a modified TF-IDF, that assigned the important words in the query to specific weights according to their parts of speech. This study changed the way of calculating the term 'frequency' by assigning a higher weight for the most important word, which is a noun or verb.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Mohammed

Omar

2020

PLoS ONE

View full text Add to dashboard Cite

The assessment of examination questions is crucial in educational institutes since examination is one of the most common methods to evaluate students' achievement in specific course. Therefore, there is a crucial need to construct a balanced and high-quality exam, which satisfies different cognitive levels. Thus, many lecturers rely on Bloom's taxonomy cognitive domain, which is a popular framework developed for the purpose of assessing students' intellectual abilities and skills. Several works have been proposed to automatically handle the classification of questions in accordance with Bloom's taxonomy. Most of these works classify questions according to specific domain. As a result, there is a lack of technique of classifying questions that belong to the multi-domain areas. The aim of this paper is to present a classification model to classify exam questions based on Bloom's taxonomy that belong to several areas. This study proposes a method for classifying questions automatically, by extracting two features, TFPOS-IDF and word2vec. The purpose of the first feature was to calculate the term frequency-inverse document frequency based on part of speech, in order to assign a suitable weight for essential words in the question. The second feature, pre-trained word2vec, was used to boost the classification process. Then, the combination of these features was fed into three different classifiers; K-Nearest Neighbour, Logistic Regression, and Support Vector Machine, in order to classify the questions. The experiments used two datasets. The first dataset contained 141 questions, while the other dataset contained 600 questions. The classification result for the first dataset achieved an average of 71.1%, 82.3% and 83.7% weighted F1-measure respectively. The classification result for the second dataset achieved an average of 85.4%, 89.4% and 89.7% weighted F1-measure respectively. The finding from this study showed that the proposed method is significant in classifying questions from multiple domains based on Bloom's taxonomy.

show abstract

Section: Modified Tf-idf (Term Weighting Methods Tfpos-idf)mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Mohammed

Omar

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…The algorithm generates Q W via the POS Weighted TF-IDF Algorithm by assigning each term in Q a weight value based on the query term's part of speech (POS), or based on the user's indication of the importance of each term if possible [15]. arithmetic mean, which is commonly used in comparable algorithms.…”

Section: Main Algorithmmentioning

confidence: 99%

A Feedback-Based Self-Organizing Query Structure Optimization Algorithm

2014

2014 Tenth International Conference on Computational Intelligence and Security

Self Cite

View full text Add to dashboard Cite

Query is one of the most important factors that can directly influence the results of information retrieval (IR). However, the query is defined by the user and thus inevitably has the following two problems: (1) the user often cannot exactly represent their search intention via query terms; (2) the user cannot effectively select the weight of each query term based on its importance toward the query's meaning. The above two problems cause the two types of uncertainty of a query. In this paper, we define them as the uncertainty of the query structure and the uncertainty of the query parameter, respectively. To eliminate the above two types of uncertainty and solve the above two problems, this paper proposes a new algorithm which includes two parts: (1) a self-organizing query structure loop which expands the initial query by adding only one term within each loop based on feedback technology until it meets the terminating condition of expansion defined by the author, and (2) an optimization algorithm based on a genetic algorithm (GA) that optimizes the weights of the expanded query vector within each loop. This algorithm provides a method of finding the optimal number of query expansion terms and improving the precision and recall of the search results. The experiment results show the effectiveness of the proposed algorithm.

show abstract

“…TF.IDF yöntemi arama motorlarının alt yapısında bulunan terimlerin sınıf içi ve sınıflar arası ağırlıklanmasını sağlayarak terimi diğer terimlerden ayıran bir terim ağırlıklandırma yöntemidir [2]. Bu geleneksel yöntemin zaman içinde farklı varyasyonları denenmiş ve daha iyi sonuçlar alınmıştır [3].…”

Section: Introductionunclassified

Two new feature extraction methods for text classification: TESDF and SADF

Kılıç

Ateş

Karakaya

et al. 2015

2015 23nd Signal Processing and Communications Applications Conference (SIU)

View full text Add to dashboard Cite

Özetçe-Bu çalışmada, metin madenciliği yöntemlerinde sıklıkla kullanılan terim frekansı-ters doküman frekansı (TF-IDF) metoduna dayanan iki yeni metin ağırlıklandırma yöntemi TF.TESDF ve TF.SADF önerilmiştir. Ayrıca önişlem aşamasında yeni bir yöntem olarak "metin sınıflandırmada fiillerin önemsizliği" yaklaşımları ortaya konulmuş ve test edilmiştir. Önerilen yöntemlerin diğer TF-IDF yöntemlerinden daha iyi sonuçlar verdiği görülmüştür. Ön işlem için kullanılan metinlerden fiillerin atılması yöntemi ile elde edilen başarının neredeyse hiç değişmediği, işlenecek verinin ortalama 1/6 oranında azaldığı gözlemlenmiştir. AnahtarKelimeler -metin sınıflandırma, terim ağırlıklandırma, ters doküman frekansı. Abstract -In this study, two new document weighting methods are proposed based on term frequency-inverse document frequency (TF-IDF) generally used in text mining methods. Also, insignificance of the verb in text classification which will be a new method in pre-processing have been put forward and tested. The better results were observed through using these methods when these methods compare with other method, It was observed that the performance rate hardly change and the data size which was processed decreased by omitting verbs of texts.

show abstract

POS weighted TF-IDF algorithm and its application for an MOOC search engine

Cited by 17 publications

References 7 publications

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

A Feedback-Based Self-Organizing Query Structure Optimization Algorithm

Two new feature extraction methods for text classification: TESDF and SADF

Contact Info

Product

Resources

About