Developing a Service for Collecting and Analyzing Electives Reviews

Abstract

This article provides a review of publications on the analysis of students’ satisfaction with the educational process based on natural language processing methods. 197 student feedback on 129 elective disciplines at University of Tyumen was collected. A comparative analysis of keyword extraction methods was conducted: statistical TF-IDF, RAKE and YAKE; contextual KeyBERT; graph-based TextRank. On the collected reviews, grouped by elective disciplines, the RAKE method had the highest F1 BERTScore with 79 %. By parsing open sources, a dataset with 2210 Russian-language reviews for courses of different educational platforms was formed. Machine learning methods for sentiment analysis were described: support vector machines, logistic regression and based on Transformers, comparison on the manually marked part of the collected reviews. After fine-tuning on the rubert-base-cased model macro-averaged F1- score became 71.6 %. Classification into three classes (negative, neutral, positive) is not performed for the whole text of the review, but separately for each sentence from that text. The implementation of a database and information system for collecting and analyzing student feedback on the studied elective courses are presented. The model for sentiment analysis of the feedback is put into a separate microservice, which is communicated through an interface of the freely distributed Python framework FastAPI. The information system is designed to help students choose electives based on more qualitative data, and teachers and university administration ‑ to draw conclusions for further transformation of the educational space, taking into account students’ opinions.