Virtual Learning Environments are growing in importance as fast as e-learning is becoming highly demanded by universities and students all over the world. This paper investigates how to automatically evaluate User eXperience in this domain using Sentiment Analysis techniques. For this purpose, a corpus with the opinions given by a total of 583 users (107 English speakers and 476 Spanish speakers) about three Learning Management Systems in different courses has been built. All the collected opinions were manually labeled with polarity information (positive, negative or neutral) by three human annotators, both at the whole opinion and sentence levels. We have applied our state-of-the-art sentiment analysis models, trained with a corpus of a different semantic domain (a Twitter corpus), to study the use of cross-domain models for this task. Cross-domain models based on Deep Neural Networks (Convolutional Neural Networks, Transformer Encoders and Attentional BLSTM models) have been tested. In order to contrast our results, three commercial systems for the same task (MeaningCloud, Microsoft Text Analytics and Google Cloud) were also tested. The obtained results are very promising and they give an insight to keep going the research of applying sentiment analysis tools on User eXperience evaluation. This is a pioneering idea to provide a better and accurate understanding on human needs in the interaction with Virtual Learning Environments and a step towards the development of automatic tools that capture the feed-back of user perception for designing Virtual Learning Environments centered in user's emotions, beliefs, preferences, perceptions, responses, behaviors and accomplishments that occur before, during and after the interaction.
The COVID-19 crisis increased the number of users of university online teaching, enhancing the importance of this learning format. Additionally, ISO 9241-210:2019 standard sets the international standards for the design of products, services and interaction systems from usability, accessibility, and user experience (User eXperience - UX) perspective. Then, in order to design interfaces and learning experiences that include motivations, feelings and needs of end users, it is necessary to previously evaluate the UX of these environments, with less general and/or laborious methods than those that currently exist. Therefore, this work aims to establish the basis of a method that allows to automatically evaluate the UX of online teaching platforms by analyzing the users' sentiment about specific aspects of their virtual learning experience. To do this, 2,035 users were surveyed about their online learning experience with a questionnaire and an open text field to give their opinion. The population surveyed were online postgraduate students of the Universitat de València and the Universidad Rey Juan Carlos, and university students of massive open online courses of the Universitat Politècnica de València. The opinions collected in Spanish from 476 students were processed with the commercial sentiment analysis and natural language processing tool MeaningCloud, to analyze the sentiment (positive, negative, or neutral) about aspects of their experience. The results present a new model that, on the one hand, ontologically classifies categories and aspects of online education with sentiment analysis techniques, and on the other hand, the model groups these categories according to UX criteria, presenting its own classification to facilitate the evaluation of online learning experiences in a concrete and automatic way. Keywords: user experience, UX, e-learning, virtual learning, sentiment analysis, data mining, MeaningCloud, natural language processing, university online learning, user centered design, UCD, NLP, VLE
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.