Bloom’s Learning Outcomes’ Automatic Classification Using LSTM and Pretrained Word Embeddings

Shaikh, Sarang; Daudpotta, Sher Muhammad; Imran, Ali Shariq

doi:10.1109/access.2021.3106443

Cited by 31 publications

(18 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the optimal variants of TF-IDF are not studied well and deeply in exam question classification. It is imperative to use the optimal variant of TF-IDF as a baseline term weighting in order to compare effectively with the improved term weighting schemes or advanced models such as word embedding and deep neural network [42,35]. This is to ensure results are more conclusive.…”

Section: Related Work In Exam Question Classificationmentioning

confidence: 99%

Identifying Optimal Baseline Variant of Unsupervised Term Weighting in Question Classification Based on Bloom Taxonomy

Sangodiah

San

Fui

et al. 2022

mendel

View full text Add to dashboard Cite

Examination is one of the common ways to evaluate the students’ cognitive levels in higher education institutions. Exam questions are labeled manually by educators in accordance with Bloom’s taxonomy cognitive domain. To ease the burden of the educators, several past research works have proposed the automated question classification based on Bloom’s taxonomy using the machine learning technique. Feature selection, feature extraction and term weighting are common ways to improve the accuracy of question classification. Commonly used term weighting method in the past work is unsupervised namely TF and TF-IDF. There are several variants of TF and TFIDF and the most optimal variant has yet to be identified in the context of question classification based on BT. Therefore, this paper aims to study the TF, TF-IDF and normalized TF-IDF variants and identify the optimal variant that can enhance the exam question classification accuracy. To investigate the variants two different classifiers were used, which are Support Vector Machine (SVM) and Naïve Bayes. The average accuracies achieved by TF-IDF and normalized TF-IDF variants using SVM classifier were 64.3% and 72.4% respectively, while using Naïve Bayes classifier the average accuracies for TF-IDF and normalized TF-IDF were 61.9% and 63.0% respectively. Generally, the normalized TF-IDF variants outperformed TF and TF-IDF variants in accuracy and F1-measure respectively. Further statistical analysis using t-test and Wilcoxon Signed also shows that the differences in accuracy between normalized TF-IDF and TF, TF-IDF are significant. The findings from this study show that the Normalized TF-IDF3 variant recorded the highest accuracy of 74.0% among normalized TF-IDF variants. Also, the differences in accuracy between Normalized TF-IDF3 and other normalized variants are generally significant, thus the optimal variant is Normalized TF-IDF3. Therefore, the normalized TF-IDF3 variant is important for benchmarking purposes, which can be used to compare with other term weighting techniques in future work.

show abstract

Section: Related Work In Exam Question Classificationmentioning

confidence: 99%

Identifying Optimal Baseline Variant of Unsupervised Term Weighting in Question Classification Based on Bloom Taxonomy

Sangodiah

San

Fui

et al. 2022

mendel

View full text Add to dashboard Cite

show abstract

“…Shaik at al. [15] have developed a text classification model that classifies the course learning outcomes (CLOs) and assessment texts into a predefined class of Bloom's taxonomy, contributing to the education domain. This model uses Skip-gram word embedding technique and LSTM classifier to perform MTC, which provides an accuracy of 87% on CLOs and 74% on assessment texts.…”

Section: Literature Surveymentioning

confidence: 99%

BERT-Based Hybrid RNN Model for Multi-class Text Classification to Study the Effect of Pre-trained Word Embeddings

Shreyashree¹,

Sunagar²,

Rajarajeswari³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Due to the Covid-19 pandemic which started in the year 2020, many nations had imposed lockdown to curb the spread of this virus. People have been sharing their experiences and perspectives on social media on the lockdown situation. This has given rise to increased number of tweets or posts on social media. Multi-class text classification, a method of classifying a text into one of the pre-defined categories, is one of the effective ways to analyze such data that is implemented in this paper. A Covid-19 dataset is used in this work consisting of fifteen predefined categories. This paper presents a multi-layered hybrid model, LSTM followed by GRU, to integrate the benefits of both the techniques. The advantages of word embeddings techniques like GloVe and BERT have been implemented and found that, for three epochs, the transfer learning based pre-trained BERThybrid model performs one percent better than GloVe-hybrid model but the state-of-the-art, fine-tuned BERT-base model outperforms the BERT-hybrid model by three percent, in terms of validation loss. It is expected that, over a larger number of epochs, the hybrid model might outperform the fine-tuned model.

show abstract

“…Several models based on RNNs have been proposed. Shaikh et al (2021) adopted an LSTM model to classify learning outcomes and questions. They manually tagged data set of Sukkur IBA University.…”

Section: Related Workmentioning

confidence: 99%

An explainable attention-based bidirectional GRU model for pedagogical classification of MOOCs

Sebbaq

Faddouli

2022

ITSE

View full text Add to dashboard Cite

Purpose The purpose of this study is, First, to leverage the limitation of annotated data and to identify the cognitive level of learning objectives efficiently, this study adopts transfer learning by using word2vec and a bidirectional gated recurrent units (GRU) that can fully take into account the context and improves the classification of the model. This study adds a layer based on attention mechanism (AM), which captures the context vector and gives keywords higher weight for text classification. Second, this study explains the authors’ model’s results with local interpretable model-agnostic explanations (LIME). Design/methodology/approach Bloom's taxonomy levels of cognition are commonly used as a reference standard for identifying e-learning contents. Many action verbs in Bloom's taxonomy, however, overlap at different levels of the hierarchy, causing uncertainty regarding the cognitive level expected. Some studies have looked into the cognitive classification of e-learning content but none has looked into learning objectives. On the other hand, most of these research papers just adopt classical machine learning algorithms. The main constraint of this study is the availability of annotated learning objectives data sets. This study managed to build a data set of 2,400 learning objectives, but this size remains limited. Findings This study’s experiments show that the proposed model achieves highest scores of accuracy: 90.62%, F1-score and loss. The proposed model succeeds in classifying learning objectives, which contain ambiguous verb from the Bloom’s taxonomy action verbs, while the same model without the attention layer fails. This study’s LIME explainer aids in visualizing the most essential features of the text, which contributes to justifying the final classification. Originality/value In this study, the main objective is to propose a model that outperforms the baseline models for learning objectives classification based on the six cognitive levels of Bloom's taxonomy. In this sense, this study builds the bidirectional GRU (BiGRU)-attention model based on the combination of the BiGRU algorithm with the AM. This study feeds the architecture with word2vec embeddings. To prove the effectiveness of the proposed model, this study compares it with four classical machine learning algorithms that are widely used for the cognitive classification of text: Bayes naive, logistic regression, support vector machine and K-nearest neighbors and with GRU. The main constraint related to this study is the absence of annotated data; there is no annotated learning objective data set based on Bloom’s taxonomy's cognitive levels. To overcome this problem, this study seemed to have no choice but to build the data set.

show abstract

Bloom’s Learning Outcomes’ Automatic Classification Using LSTM and Pretrained Word Embeddings

Cited by 31 publications

References 43 publications

Identifying Optimal Baseline Variant of Unsupervised Term Weighting in Question Classification Based on Bloom Taxonomy

Identifying Optimal Baseline Variant of Unsupervised Term Weighting in Question Classification Based on Bloom Taxonomy

BERT-Based Hybrid RNN Model for Multi-class Text Classification to Study the Effect of Pre-trained Word Embeddings

An explainable attention-based bidirectional GRU model for pedagogical classification of MOOCs

Contact Info

Product

Resources

About