Cross-Domain Polarity Models to Evaluate User eXperience in E-learning

Abstract

Virtual Learning Environments are growing in importance as fast as e-learning is becoming highly demanded by universities and students all over the world. This paper investigates how to automatically evaluate User eXperience in this domain using Sentiment Analysis techniques. For this purpose, a corpus with the opinions given by a total of 583 users (107 English speakers and 476 Spanish speakers) about three Learning Management Systems in different courses has been built. All the collected opinions were manually labeled with polarity information (positive, negative or neutral) by three human annotators, both at the whole opinion and sentence levels. We have applied our state-of-the-art sentiment analysis models, trained with a corpus of a different semantic domain (a Twitter corpus), to study the use of cross-domain models for this task. Cross-domain models based on Deep Neural Networks (Convolutional Neural Networks, Transformer Encoders and Attentional BLSTM models) have been tested. In order to contrast our results, three commercial systems for the same task (MeaningCloud, Microsoft Text Analytics and Google Cloud) were also tested. The obtained results are very promising and they give an insight to keep going the research of applying sentiment analysis tools on User eXperience evaluation. This is a pioneering idea to provide a better and accurate understanding on human needs in the interaction with Virtual Learning Environments and a step towards the development of automatic tools that capture the feed-back of user perception for designing Virtual Learning Environments centered in user's emotions, beliefs, preferences, perceptions, responses, behaviors and accomplishments that occur before, during and after the interaction.

Examining User Experience of eLearning Systems using EKhool Learners

“…The comparative methods used for the analysis of the proposed user evaluation method based on RideNN are Nielsen"s heuristics & didactic metrics [1], multidimensional knowledge graph framework [2], Cross-Domain Polarity Model based on DNN [3], and User experience based Adaptive E-Learning Hypermedia System (U-AEHS) [4].…”

“…Sanchis-Font, R.et al [3] modelled a Cross-domain models for the eLearning systems based on the deep neural networks. In this method, the human annotators were used for labeling the opinions collected with the polarity information.…”

“…The challenges faced during the examination of the user experience of the eLearning systems are as follows:  The challenge in the Cross-domain models was the development of the virtual learning environments that determined the aspects that are relevant and had influence on the students. The questionnaire should also contain large amount of data for training the model [3].  The AEHS method had high overall mean score but the challenge lies in expanding the area reflecting the method.…”

Attention-based Approaches for Text Analytics in Social Media and Automatic Summarization

“…Also, the overwhelming expansion of individuals and organizations in social networks was a key factor, thus growing the necessity to extract insights from the users' opinions in this kind of platforms. The interest in sentiment analysis is not focused on a specific industrial sector, but rather it is scattered across all the industry, being used in a plethora of relevant environments such as: political tendency identification [102], user experience evaluation [13,14], consumer confidence and political opinion [103], election prediction [104], prediction of box-office revenues for movies [105] or stock market prediction [106]. Although sentiment analysis as discussed in this introduction is focused in sentiment classification, it is convenient to highlight that there are many other research lines where sentiment is one of the fundamental pillars, and almost all of them remain unexplored, due to the targeting of the research community in the sentiment classification.…”

“…In addition to the TASS, that is the reference workshop for training and evaluating sentiment analysis approaches in Spanish, many other works explore sentiment analysis in other environments where the Spanish language is used: opinion analysis of microblogging data [128], deception detection [129], user experience evaluation [13], voting intention inference [130], marijuana infodemiology [131], financial analysis [132], early detection of infectious diseases [133], analysis of medical opinions [134], analysis of user reviews about restaurant and hotels [135], products [136], and analysis of Spanish online videos [137].…”