A Comparative Analysis On Term Weighting In Exam Question Classification

Sangodiah, Anbuselvan; Fui, Yong Tien; Heng, Lim Ean; Jalil, Norazira A.; Ayyasamy, Ramesh Kumar; Meian, Kok Hui

doi:10.1109/ismsit52890.2021.9604639

Cited by 3 publications

(17 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Stanford tagger (version 4.2.0) was used to obtain the POS tagging of each term after tokenization and punctuation removal. Stanford tagger was also used in past studies [17,24]. Following that, the stop words in the question were removed using NLTK's standard stop word list.…”

Section: Preprocessingmentioning

confidence: 99%

“…Unigram: Unigram is a simple feature extraction method that creates a set of all unique terms from the dataset. Several Past studies [16,17,24] used unigram to obtain the feature set from the questions. Other than unigram, as reported by [24], there are many more techniques to obtain the feature set, such as bigram, trigram, POS tagging, headword, and so forth.…”

Section: Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

USTW Vs. STW: A Comparative Analysis for Exam Question Classification based on Bloom’s Taxonomy

Gani

Ayyasamy

Sangodiah

et al. 2022

mendel

View full text Add to dashboard Cite

Bloom’s Taxonomy (BT) is widely used in educational institutions to produce high-quality exam papers to evaluate students’ knowledge at different cognitive levels. However, manual question labeling takes a long time, and not all evaluators are familiar with BT. The researchers worked to automate the exam question classification process based on BT as a solution. Enhancement in term weighting is one of the ways to increase classification accuracy while working with text data. However, all the past work on the term weighting in exam question classification focused on unsupervised term weighting (USTW) schemes. The supervised term weighting (STW) schemes showed effectiveness in text classification but were not addressed in past studies of exam question classification. As a result, this study focused on the effectiveness of STW in classifying exam questions using BT. Hence, this research performed a comparative analysis between the USTW schemes and STW for exam question classification. The STW schemes used in this study are TF-ICF, TF-IDF-ICF, and TF-IDF-ICSDF, whereas the USTW schemes used for comparison are TF-IDF, ETF-IDF, and TFPOS-IDF. This study used Support Vector Machines (SVM), Na¨ıve Bayes (NB), and Multilayer Perceptron (MLP) to train the model. Accuracy and F1 score were used in this study to evaluate the classification result. The experiment result showed that overall, the STW scheme TF-ICF outperformed all the other schemes, followed by the USTW scheme ETF-IDF. Both the ETF-IDF and TFPOS-IDF outperformed standard TFIDF. The outcome of this study indicates the future research direction where the combination of STW and USTW schemes may increase the Accuracy of BT-based exam question classification.

show abstract

Section: Preprocessingmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

USTW Vs. STW: A Comparative Analysis for Exam Question Classification based on Bloom’s Taxonomy

Gani

Ayyasamy

Sangodiah

et al. 2022

mendel

View full text Add to dashboard Cite

show abstract

“…The punctuations present in the question were removed and tokenized the questions. After that, the pos tagging was applied to the terms using the Stanford tagger (version 4.2.0) [26] by following past studies [11], [15], [27]. The BT verbs need to identify to use later in the proposed scheme of this study.…”

Section: B Preprocessingmentioning

confidence: 99%

“…SVM has been widely used in text and examination question classification [19], [21], [32]. The past studies [15], [27] of examination question classification used the linear kernel of SVM, also known for higher accuracy in text classification [33]. Hence, this study used the linear kernel of SVM with the default settings of Scikit-learn.…”

Section: Classification and Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

ETFPOS-IDF: A Novel Term Weighting Scheme for Examination Question Classification Based on Bloom’s Taxonomy

et al. 2022

Self Cite

View full text Add to dashboard Cite

Numerous earlier studies focused on the term weighting scheme to increase examination question classification accuracy based on Bloom's Taxonomy (BT). While determining the cognitive level of the examination question, all the terms present in the question are not equally significant. Verbs are the most important parts of speech while assigning weights to the terms. However, two types of verbs may be present in the questions: BT and supporting. BT verbs have a higher impact on determining the cognitive level of a question than supporting verbs. Nevertheless, the proposed schemes of past studies assigned equal weight to both types of verbs. Therefore, this study aims to introduce the term weighting scheme ETFPOS-IDF, which assigns BT a higher weight than supporting verbs. The BT verbs were identified based on their position in the questions. Three datasets and three classifiers: Support Vector Machine, Artificial Neural Network, and Random Forest, were used in this study. Two evaluation metrics: accuracy and F1 score, were used to evaluate the performance of the proposed model. The experiment results showed that the proposed ETFPOS-IDF outperformed all the schemes introduced by earlier studies in examination question classification and achieved 0.749 in accuracy and 0.746 in F1 score. The finding of this study demonstrates that distinguishing between different verb types is significant in reducing the misclassification of examination questions. This research contributed by introducing a novel term weighting scheme in classifying examination questions based on BT. Future work may involve identifying the optimal weight for both types of verbs, evaluating the proposed scheme with a larger dataset, and comparing the performance with deep learning.

show abstract

Bloom’s Taxonomy-based exam question classification: The outcome of CNN and optimal pre-trained word embedding technique

et al. 2023

View full text Add to dashboard Cite

A Comparative Analysis On Term Weighting In Exam Question Classification

Cited by 3 publications

References 18 publications

USTW Vs. STW: A Comparative Analysis for Exam Question Classification based on Bloom’s Taxonomy

USTW Vs. STW: A Comparative Analysis for Exam Question Classification based on Bloom’s Taxonomy

ETFPOS-IDF: A Novel Term Weighting Scheme for Examination Question Classification Based on Bloom’s Taxonomy

Bloom’s Taxonomy-based exam question classification: The outcome of CNN and optimal pre-trained word embedding technique

Contact Info

Product

Resources

About