A Forward-Selection Algorithm for SVM-Based Question Classification in Cognitive Systems

Pota, Marco; Esposito, Massimo; Pietro, Giuseppe De

doi:10.1007/978-3-319-39345-2_52

Cited by 13 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea behind SVM is to find an appropriate hyperplane that best split-up the two sets of data from each other, by maximizing the margin between the hyperplane and the set of data points nearest to it. In-text classification SVM with linear kernel perform well [54]. Therefore, this study will use it.…”

Section: Support Vector Machine (Svm)mentioning

confidence: 99%

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Mohammed

Omar

2020

PLoS ONE

View full text Add to dashboard Cite

The assessment of examination questions is crucial in educational institutes since examination is one of the most common methods to evaluate students' achievement in specific course. Therefore, there is a crucial need to construct a balanced and high-quality exam, which satisfies different cognitive levels. Thus, many lecturers rely on Bloom's taxonomy cognitive domain, which is a popular framework developed for the purpose of assessing students' intellectual abilities and skills. Several works have been proposed to automatically handle the classification of questions in accordance with Bloom's taxonomy. Most of these works classify questions according to specific domain. As a result, there is a lack of technique of classifying questions that belong to the multi-domain areas. The aim of this paper is to present a classification model to classify exam questions based on Bloom's taxonomy that belong to several areas. This study proposes a method for classifying questions automatically, by extracting two features, TFPOS-IDF and word2vec. The purpose of the first feature was to calculate the term frequency-inverse document frequency based on part of speech, in order to assign a suitable weight for essential words in the question. The second feature, pre-trained word2vec, was used to boost the classification process. Then, the combination of these features was fed into three different classifiers; K-Nearest Neighbour, Logistic Regression, and Support Vector Machine, in order to classify the questions. The experiments used two datasets. The first dataset contained 141 questions, while the other dataset contained 600 questions. The classification result for the first dataset achieved an average of 71.1%, 82.3% and 83.7% weighted F1-measure respectively. The classification result for the second dataset achieved an average of 85.4%, 89.4% and 89.7% weighted F1-measure respectively. The finding from this study showed that the proposed method is significant in classifying questions from multiple domains based on Bloom's taxonomy.

show abstract

Section: Support Vector Machine (Svm)mentioning

confidence: 99%

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Mohammed

Omar

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…The aim of SVM is to find the favourable hyperplane that separates two sets of data from each other, by maximizing the separation margin among the hyperplane and the set of data points closest to it. This study uses SVM with linear kernel since it is popular in text classification [22]. Fig.…”

Section: Classificationmentioning

confidence: 99%

Question Classification Based on Bloom’s Taxonomy Using Enhanced TF-IDF

Mohammed¹,

Omar²

2018

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

Bloom's Taxonomy has been used widely in the educational environment to measure, evaluate and write high-quality exams. Therefore, many researchers have worked on the automation for classification of exam questions based on Bloom's Taxonomy. The aim of this study is to make an enhancement for one of the most popular statistical feature, which is TF-IDF, to improve the performance of exam question classification in accordance to Bloom's Taxonomy cognitive domain. Verbs play an important role in determining the level of a question in Bloom's Taxonomy. Thus, the improved method assigns the impact factor for the words by taking the advantage of the part-of-speech tagger. The higher impact factor assigns to the verbs, then to the noun and adjective, after that, the lower impact factor assigns to the other part-of-speech. The dataset that has been used in this study is consist of 600 questions, divided evenly into each Bloom level. The questions first pass into the preprocessing phase in which they are prepared to be suitable for applying the proposed enhanced feature. For classification purpose, three machine learning classifiers are used Support Vector Machine, Naïve Bayes, and K-Nearest Neighbour. The enhanced feature shows satisfactory result by outperforming the classical feature TF-IDF via all classifiers in terms of weighted recall, precision, and F1-measure. On the other hand, Support Vector Machine has superior performance over other classifiers Naïve Bayes, and K-Nearest Neighbour by achieving an average of 86%, 85%, and 81.6% weighted F1-measure respectively. However, these results are promising and encouraging for further investigations.

show abstract

“…SVM has been widely used in text and examination question classification [19], [21], [32]. The past studies [15], [27] of examination question classification used the linear kernel of SVM, also known for higher accuracy in text classification [33]. Hence, this study used the linear kernel of SVM with the default settings of Scikit-learn.…”

Section: Classification and Evaluationmentioning

confidence: 99%

ETFPOS-IDF: A Novel Term Weighting Scheme for Examination Question Classification Based on Bloom’s Taxonomy

et al. 2022

View full text Add to dashboard Cite

Numerous earlier studies focused on the term weighting scheme to increase examination question classification accuracy based on Bloom's Taxonomy (BT). While determining the cognitive level of the examination question, all the terms present in the question are not equally significant. Verbs are the most important parts of speech while assigning weights to the terms. However, two types of verbs may be present in the questions: BT and supporting. BT verbs have a higher impact on determining the cognitive level of a question than supporting verbs. Nevertheless, the proposed schemes of past studies assigned equal weight to both types of verbs. Therefore, this study aims to introduce the term weighting scheme ETFPOS-IDF, which assigns BT a higher weight than supporting verbs. The BT verbs were identified based on their position in the questions. Three datasets and three classifiers: Support Vector Machine, Artificial Neural Network, and Random Forest, were used in this study. Two evaluation metrics: accuracy and F1 score, were used to evaluate the performance of the proposed model. The experiment results showed that the proposed ETFPOS-IDF outperformed all the schemes introduced by earlier studies in examination question classification and achieved 0.749 in accuracy and 0.746 in F1 score. The finding of this study demonstrates that distinguishing between different verb types is significant in reducing the misclassification of examination questions. This research contributed by introducing a novel term weighting scheme in classifying examination questions based on BT. Future work may involve identifying the optimal weight for both types of verbs, evaluating the proposed scheme with a larger dataset, and comparing the performance with deep learning.

show abstract

A Forward-Selection Algorithm for SVM-Based Question Classification in Cognitive Systems

Cited by 13 publications

References 16 publications

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

Question Classification Based on Bloom’s Taxonomy Using Enhanced TF-IDF

ETFPOS-IDF: A Novel Term Weighting Scheme for Examination Question Classification Based on Bloom’s Taxonomy

Contact Info

Product

Resources

About