Identifying engagement patterns with video annotation activities: A case study in professional development

Abstract

The rapid growth of blended and online learning models in higher education has resulted in a parallel increase in the use of audio-visual resources among students and teachers. Despite the heavy adoption of video resources, there have been few studies investigating their effect on learning processes and even less so in the context of academic development. This paper uses learning analytic techniques to examine how academic teaching staff engage with a set of prescribed videos and video annotations in a professional development course. The data was collected from two offerings of the course at a large research-intensive university in Australia. The data was used to identify patterns of activity and transition states as users engaged with the course videos and video annotations. Latent class analysis and hidden Markov models were used to characterise the evolution of engagement throughout the course. The results provide a detailed description of the evolution of learner engagement that can be readily translated into action aimed at increasing the quality of the learning experience.

The trajectory of early disengagement: A longitudinal learning analytics study over a full program

“…Our RQ1 sought to identify the course-level engagement states of the students over a full program using LCA clustering method. While few studies have looked into a full program, the three engagement states at the course level identi ed in our study were in line with the existing literature [9,24,45]. Our three identi ed engagement clusters were: 1) an actively engaged cluster similar to the intense, highly active groups reported by others [9,24,45].…”

“…The three clusters they identi ed were 1) consistent: who are engaged are more likely to remain engaged, 2) get-it-done: assessmentoriented but still able to follow up with the program, 3) disorganized: who are mostly disengaged. Similar results were reported by [24], who identi ed three trajectories along a professional development course.…”

Combining self-reported and observational measures to assess university student academic performance in blended course designs

“…Second, the observational measures in the study captured only the frequency of students' online interactions with the online learning tasks, which does not represent the complexity of students' online interactions. Future research should aim to use other observational measures in addition to frequency, including but not limited to duration of the online interaction and the time-stamped trace data of sequences of the online interaction (Mirriahi et al, 2018;Winne et al, 2017). Through applying advanced process-mining methods to analysing these types of digital trace data, the more dynamic nature of students' online learning can be revealed (Jovanović et al, 2017;Sonnenberg & Bannert, 2019).…”

Worked Example Videos for Blended Learning in Undergraduate Engineering

“…One opportunity for capitalizing on the popularity of WEVs within undergraduate engineering is by enhancing how students engage with content outside of class. According to the theory of selfregulated learning, students regulate their learning by continuously evaluating the quality of their learning products (Mirriahi et al 2018). In the context of self-directed study in math-heavy engineering courses, students are often given homework exercises with written solutions to assist this process.…”

“…This enables students to compare their solutions to the model ones, and reason with themselves on whether they have grasped the concepts (Belski 2011). However, a major influence on the quality of this metacognitive monitoring is prior knowledge (Mirriahi et al 2018). As written solutions are unable to effectively convey the underlying problem-solving strategies and thought processes used to develop a solution, students must infer from the lines of working why the process has been done a certain way.…”