A Sentiment Analysis Model for Faculty Comment Evaluation Using Ensemble Machine Learning Algorithms

Lalata, Jay-ar P.; Gerardo, Bobby D.; Medina, Ruji P.

doi:10.1145/3341620.3341638

Cited by 29 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are also interested in adopting a visualization to condense all peer reviews into an understandable graphic (perhaps using aspectlevel opinion mining) to increase the information presented to an instructor for grading confidence. unique (15), outstand (11), creative( 9) hard(3), need(3), short(3) GM 10% unique (25), creative (19), structure (18), example (18) short (14), hard (7), need(5) GM 15% example (37), unique (37), impressive (32) hard (16), need (15), short(15) GM 20% impressive (60), example (57), unique (54) hard (21), need (21), short (21) SE 1% fantastic(3), engage(2), enjoy(2), useful(2), outstand(2) punctuation(1), incorrect(1), awkward(1), heavy(1), bor(1), stuck(1) SE 5% useful (15), outstand (10), enjoy (9), unique (9), fantastic( 9) need( 10), short (6), add(3), bor(3), heavy(3), hard(3) SE 10% useful (25), example (25), enjoy (18), outstand( 18) need( 21), short (16), hard(11) SE 15% example (52), useful (32), outstand (30) need (36), short (24), hard(15) SE 20% example(87), understand (40), useful (40) need (48), short…”

Section: Discussionmentioning

confidence: 99%

“…• student attrition over time in three MOOCs (captured from course forums and scored by a product reviews lexicon [28]) or predicted attrition in a single MOOC (captured from a course forum and scored by SentiWord-Net 3.0 as one feature of a neural network [29]); • the mood of students toward a teacher (captured via Twitter and scored by Naïve Bayes [30]) or as an "emotional thermometer for teaching" in virtual classrooms (captured in forum posts and scored by an ensemble [31]); • negative students or course issues (captured in online course forums and scored by the Microsoft Text Analytics API [32] or captured from social media and scored by a mixed graph of terms [33]); • teacher strengths and weaknesses identified by students (a proposed system with sentiment captured via questionnaire and scored by Naïve Bayes [34], or a proposed multilingual system with sentiment captured from Coursera peer reviews and scored by a lexicon in R [35], or a system with sentiment captured from teacher evaluations and scored by an ensemble [36]); • student perception of internship experience (captured from transcribed interviews and scored manually [37]); • an alternative way to view poetry and a means of student discussion on the relationship between text and numbers (captured from a Walt Whitman poem and scored by a proprietary sentiment analysis tool [38]).…”

Section: B Sentiment Analysis In Educationmentioning

confidence: 99%

See 1 more Smart Citation

Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment

Beasley¹,

Piegl²,

Rosen³

2021

Preprint

View full text Add to dashboard Cite

Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To date, no work has 1) leveraged sentiment analysis in the peer-review process to inform or validate grades or 2) utilized aspect extraction to craft a review form from what students actually communicated. Our work utilizes, rather than discards, student data from review form comments to deliver better information to the instructor. In this work, we detail the process by which we create our domaindependent lexicon and aspect-informed review form as well as our entire sentiment analysis algorithm which provides a fine-grained sentiment score from text alone. We end by analyzing validity and discussing conclusions from our corpus of over 6800 peer reviews from nine courses to understand the viability of sentiment in the classroom for increasing the information from and reliability of grading open-ended assignments in large courses.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Sentiment Analysis In Educationmentioning

confidence: 99%

Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment

Beasley¹,

Piegl²,

Rosen³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They also use tag clouds, sentiment scores, and frequency-based filters to provide new insights into teacher's performance. An ensemble model that consists of five machine learning algorithms namely Naive Bayes, Logistic Regression, Support Vector Machine, Decision Tree, and Random Forest algorithms was used by Lalata et al [6] to analyze and identify the polarity of the written comments of the students. The result showed that there is a marginally significant relationship between the numerical rating and the overall sentiment scores.…”

Section: Introductionmentioning

confidence: 99%

Harmonizing Perceptions: Exploring Alignment of Numerical and Textual Evaluation Through Fine-Grained Sentiment Analysis

A. Baldelovar

2024

jes

View full text Add to dashboard Cite

The assessment of faculty teaching effectiveness plays a pivotal role in shaping educational practices and ensuring academic excellence. While numerical evaluations provide quantifiable measures of teaching effectiveness, textual comments offer valuable qualitative insights into the faculty's instructional methods and interactions. However, discrepancies between these evaluation formats may hinder the accurate assessment of faculty performance. This study aims to determine the alignment between the numerical and textual evaluations of faculty teaching effectiveness using fine-grained sentiment analysis and determine the relationship between numerical ratings and textual comments to identify patterns of consistency or divergence. A large dataset of anonymized BSCS students’ feedback is analyzed through sentiment analysis to extract sentiments expressed in the textual comments. The study results reveal a misalignment between students’ subjective perceptions expressed in the textual comments and the numerical ratings. This misalignment arises due to several factors, such as student communication style variations, individual interpretations of rating scales, or subjective biases. This study advances faculty evaluation methodologies and offers useful recommendations for organizations looking to improve assessment procedures as well as educational institutions that want to improve the precision and efficacy of their faculty evaluation systems for better faculty development and decision-making.

show abstract

“…Using the SVM algorithm, they achieved an accuracy score of 93.35%. Lalata et al [32] used SA to analyze student comments in the classroom. The ensemble and individual models of LR, SVM, DT, and RF algorithms were evaluated and compared on a dataset of comments of 1413 positive, 327 negative, and 82 neutral comments.…”

Section: Introductionmentioning

confidence: 99%

Sentiment analysis of student feedback using attention-based RNN and transformer embedding

Zyout,

Zyout

2024

IJ-AI

View full text Add to dashboard Cite

Sentiment analysis systems aim to assess people’s opinions across various domains by collecting and categorizing feedback and reviews. In our study, researchers put forward a sentiment analysis system that leverages three distinct embedding techniques: automatic, global vectors (GloVe) for word representation, and bidirectional encoder representations from transformers (BERT). This system features an attention layer, with the best model chosen through rigorous comparisons. In developing the sentiment analysis model, we employed a hybrid dataset comprising students’ feedback and comments. This dataset comprises 3,820 comments, including 2,773 from formal evaluations and 1,047 generated by ChatGPT and prompting engineering. Our main motivation for integrating generative AI was to balance both positive and negative comments. We also explored recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), and bidirectional long short-term memory (Bi-LSTM), with and without pre-trained GloVe embedding. These techniques produced F-scores ranging from 67% to 69%. On the other hand, the sentiment model based on BERT, particularly its KERAS implementation, achieved higher F-scores ranging from 83% to 87%. The Bi-LSTM architecture outperformed other models and the inclusion of an attention layer further enhanced the performance, resulting in F-scores of 89% and 88% from the Bi-LSTM-BERT sentiment models, respectively.

show abstract

A Sentiment Analysis Model for Faculty Comment Evaluation Using Ensemble Machine Learning Algorithms

Cited by 29 publications

References 8 publications

Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment

Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment

Harmonizing Perceptions: Exploring Alignment of Numerical and Textual Evaluation Through Fine-Grained Sentiment Analysis

Sentiment analysis of student feedback using attention-based RNN and transformer embedding

Contact Info

Product

Resources

About