Question Classification Based on Bloom’s Taxonomy Using Enhanced TF-IDF

Abstract

Bloom's Taxonomy has been used widely in the educational environment to measure, evaluate and write high-quality exams. Therefore, many researchers have worked on the automation for classification of exam questions based on Bloom's Taxonomy. The aim of this study is to make an enhancement for one of the most popular statistical feature, which is TF-IDF, to improve the performance of exam question classification in accordance to Bloom's Taxonomy cognitive domain. Verbs play an important role in determining the level of a question in Bloom's Taxonomy. Thus, the improved method assigns the impact factor for the words by taking the advantage of the part-of-speech tagger. The higher impact factor assigns to the verbs, then to the noun and adjective, after that, the lower impact factor assigns to the other part-of-speech. The dataset that has been used in this study is consist of 600 questions, divided evenly into each Bloom level. The questions first pass into the preprocessing phase in which they are prepared to be suitable for applying the proposed enhanced feature. For classification purpose, three machine learning classifiers are used Support Vector Machine, Naïve Bayes, and K-Nearest Neighbour. The enhanced feature shows satisfactory result by outperforming the classical feature TF-IDF via all classifiers in terms of weighted recall, precision, and F1-measure. On the other hand, Support Vector Machine has superior performance over other classifiers Naïve Bayes, and K-Nearest Neighbour by achieving an average of 86%, 85%, and 81.6% weighted F1-measure respectively. However, these results are promising and encouraging for further investigations.

Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec

“…However, other researchers [5,6,[20][21][22][23][24] used machine learning techniques to classify Bloom's taxonomy into cognitive levels. A neural network is used by [22] with different feature sets: whole feature, document frequency, and category frequency-document frequency to classify questions by training the model with a scaled conjugate gradient learning algorithm.…”

“…As a result, removing stop words does not make a big impact on the result. Similarly, [6] used the same dataset prepared by [20], but with enhanced TF-IDF which is multiplied with impact factor. Then the classification process handled by three classifiers NB, KNN, and SVM where SVM performance superior other classifiers.…”

“…It assigns scores to the importance of a word in a document, based on the lexical and morphological properties of the text. It is extensively used in many studies [25][26][27][28][29], and in question classification [3,6,20,23,[30][31][32] However, some researchers used TF-IDF as it is, while others proposed some enhancement to the way TF-IDF is calculated in order to improve it. This is because classical TF-IDF does not capture some useful information such as the impact of word distribution between various classes [6].…”

“…This enhancement performs better than the classical method. Similarly, [6] used the impact factor to enhance the result of TF-IDF to classify question based on Bloom's taxonomy.…”

“…Recently, researchers [5][6][7][8][9] have shown an increased interest in automating evaluating examination based on Bloom's taxonomy cognitive domain. Various techniques have been used along with different features such as lexical features, and syntactic features, while few of them used semantic features.…”