Evaluation of a learning analytics application for open edX platform

Abstract

Massive open online courses (MOOCs) have recently emerged as a revolution in education. Due to the huge amount of users, it is difficult for teachers to provide personalized instruction. Learning analytics computer applications have emerged as a solution. At present, MOOC platforms provide low support for learning analytics visualizations, and a challenge is to provide useful and effective visualization applications about the learning process. At this paper we review the learning analytics functionality of Open edX and make an overview of our learning analytics application ANALYSE. We present a usability and effectiveness evaluation of ANALYSE tool with 40 students taking a Design of Telematics Applications course. The survey obtained very positive results in a system usability scale (SUS) questionnaire (78.44/100) in terms of the usefulness of visualizations (3.68/5) and the effectiveness ratio (92/100) of the actions required for the respondents. Therefore, we can conclude that the implemented learning analytics application is usable and effective.

Applying Learning Analytics to Detect Sequences of Actions and Common Errors in a Geometry Game

“…As it is important to make sure that the visualizations are usable by teachers, we will be working on obtaining evidences of the interpretability of these visualizations and their usefulness in real classrooms, as we do not have a validation case study with teachers yet, which we consider as our main limitation. Other works have analyzed the impact of visualizations in the learning process and evaluated the usability and effectiveness of dashboards [ 49 ]. Mazza and Milani [ 50 ] revealed that graphical representations might help instructors to identify individuals that need particular attention, to discover patterns and trends in accesses and discussions, and to reflect on their teaching practice.…”

A Framework for Computerized Progress Feedback in Tertiary Education

“…Higher education institutions are increasingly implementing big data analytics tools to assist them in decisionmaking (Rubel and Jones, 2016). Ruipérez-Valiente et al (2016) found that although Learning Management Systems (LMSs) collect different forms of student data, not many provide appropriate visualizations about the learning process. The purpose of learning analytics systems can vary but to improve learning, it should assist in the decision-making process and also help with detecting problem-cases early on.…”

“…The purpose of learning analytics systems can vary but to improve learning, it should assist in the decision-making process and also help with detecting problem-cases early on. It was found that using learning analytics should be extended to not only augment the teacher's information but also provide students with selfawareness tools (Ruipérez-Valiente et al, 2016).…”

Análisis del rendimiento académico en un curso de educación superior a través de Open EDX

“…La implementación de cursos masivos sugieren un desafío instruccional complejo que debe responder a los nuevos escenarios educativos. Tal y como concluyeron Ruipérez-Valiente et al (2017), quienes analizaron la usabilidad de un curso alojado en Open EDX, cuyos usuarios estimaron positivamente el recurso y obtuvieron altas tasas de visualización. De esta manera, hay ciertos elementos coincidentes con esta investigación, en especial con las visualizaciones mostradas con el perfil de estudiantes de Ingeniería.…”