Mixing and Matching Learning Design and Learning Analytics

Abstract

Abstract. In the last five years, learning analytics has proved its potential in predicting academic performance based on trace data of learning activities. However, the role of pedagogical context in learning analytics has not been fully understood. To date, it has been difficult to quantify learning in a way that can be measured and compared. By coding the design of e-learning courses, this study demonstrates how learning design is being implemented on a large scale at the Open University UK, and how learning analytics could support as well as benefit from learning design. Building on our previous work, our analysis was conducted longitudinally on 23 undergraduate distance learning modules and their 40,083 students. The innovative aspect of this study is the availability of fine-grained learning design data at individual task level, which allows us to consider the connections between learning activities, and the media used to produce the activities. Using a combination of visualizations and social network analysis, our findings revealed a diversity in how learning activities were designed within and between disciplines as well as individual learning activities. By reflecting on the learning design in an explicit manner, educators are empowered to compare and contrast their design using their own institutional data.

Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning

“…When it comes to learning design activities (see Appendix C), most of the studies included in the review have used assimilative, assessment, communication, finding and handling of information, and productive activities, as defined in the learning design taxonomy [34]. This is also supported by Rienties and Nguyen research studies on the impact of learning design in university settings [5], [46], [86], [85]. They reported that learning design has a strong influence on learner's satisfaction [5].…”

“…In particular, communication and interactive activities engage students to spend more time in VLEs compared to productive and experiential activities [86]. Although the design of learning activities depend on the module [85], educators design learning activities differently over the time line of the course and reduce the variety of learning design activities when they introduce assessment activities [46].…”

“…On the other hand, few studies reported use of embedded analytics, that are integrated into the learning environment and could be used for real-time feedback [7], [47], [67], [71]. Furthermore, findings show that learning analytics mostly have been used to explore the physical, digital, and human elements in the learning ecosystem [42], [81], for postcourse reflection and recommendation of resources [67], for prediction [82], [83], [46], [66], as a tool [71], [77], [84], to detect weaknesses and strengths in the learning design [5], [85], [86], and to define scripts with monitoring information [47]. Another conclusion that can be draw from the selected studies is the common use of time-related learning analytics to evaluate the design of the learning activities and learners' online behavior [87], [71]; score and frequency learning analytics to evaluate performance, difficulty of the learning content and assessment strategies [63], [88]; and aggregated learning analytics to identify trends and behaviors [5], [89].…”

Using Temporal Analytics to Detect Inconsistencies Between Learning Design and Students’ Behaviours

“…Second, by capturing and visualizing the design of learning activities, the LD approach could provide a pedagogical context to support interpreting and translating LA findings into interventions (Lockyer et al, 2013;Persico & Pozzi, 2015). A gradual accumulation of empirical evidence has indicated great potential in connecting LA with LD (Gašević, Dawson, Rogers, & Gasevic, 2016;Nguyen, Rienties, & Toetenel, 2017a;Nguyen, Rienties et al, 2017b;Nguyen, Rienties, Toetenel, Ferguson, & Whitelock, 2017;Rienties & Toetenel, 2016;Rienties et al, 2015). In a large-scale study of 151 modules and their 111,256 students at a distance educational institution, for example, Rienties and Toetenel (2016) revealed significant relations between LD and VLE behaviour, along with student satisfaction and retention.…”

Collaboration Analytics Need More Comprehensive Models and Methods: An Opinion Paper

“…Although it is not related to collaborative classes, Nguyen, Rienties, and Toetenel (2017) provide an example of analyzing learning activities based on learning designs. They illustrate the variance in the time allotted for different activities over 32 course weeks in 24 modules of the Open University, UK (Nguyen et al, 2017). They start by categorizing learning activities into six types (e.g., assimilative: paying attention to information) with reference to Rienties and Toetenel (2016).…”