Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures

Abstract

The availability of large volumes of interaction data and scalable data mining techniques have made possible to study the online behaviour for millions of Web users. Part of the efforts have focused on understanding how users interact and engage with web content. However, the measurement of within-content engagement remains a difficult and unsolved task. This is because of the lack of standardised, wellvalidated methods for measuring engagement, especially in an online context. To address this gap, we perform a controlled user study where we observe how users respond to online news in the presence or lack of interest. We collect mouse tracking data, which are known to correlate with visual attention, and examine how cursor behaviour can inform user engagement measures. The proposed method does not use any pre-determined concepts to characterise the cursor patterns. We, rather, follow an unsupervised approach and use a large set of features engineered from our data to extract the cursor patterns. Our findings support the connection between gaze and cursor behaviour but also, and more importantly, reveal other dependencies, such as the correlation between cursor activity and experienced affect. Finally, we demonstrate the value of our method by predicting the outcome of online news reading experiences.

Understanding User Attention and Engagement in Online News Reading

“…The above and other works showed dwell time to be strong signal of user interest, an important component of user engagement [3,4,24]. Their aim was also to identify aspects of the webpage that make users spend time on it.…”

“…For instance, [31] found that the ratio of mouse cursor movement to time spent on a webpage was a good indicator of how interested users were in the webpage content, and explored the extent to which cursor tracking can inform about whether users are attentive to certain content when reading it, and what their experience was. Recently, [3] recorded the mouse cursor movements from users reading interesting versus noninteresting news articles, from which they generated cursor gesture patterns through unsupervised learning. They identified several significant correlations between cursor behaviour and the focused attention and affect engagement metrics, and could predict with high accuracy user interests on the news articles based on the cursor gestures.…”

Engagement in HCI

“…Turning to the comparison corpus examined in 2018, authors continue to refer to the persistent challenge of understanding and measuring engagement (Arapakis et al 2014;de Oliveira et al 2016;Drutsa et al 2015;Hamari et al 2016). Ray et al write that "[a]lthough researchers implicitly concur on the significance of engagement in the context of online communities, the notion of engagement itself remains relatively little understood in the information systems literature" (Ray et al 2014).…”

Gaze-Driven Design Insights to Amplify Debugging Skills: A Learner-Centered Analysis Approach

“…Nowadays, the functionality of many IDEs can be extended with plugins to collect data with additional features, lifting the previous technological restrictions. Examples include student-IDE interactions (Brown, Kolling, McCall, & Utting, 2014;Hundhausen, Olivares, & Carter, 2017), asynchronous discussion posts , automatic testing against test cases (Edwards & Perez-Quinones, 2008), survey/quiz data to gain insights into learner attitudes and conceptual understanding of tasks (Ihantola, Sorva, & Vihavainen, 2014), studies collecting data from eye-trackers (Busjahn et al, 2014;Kevic et al, 2015;Mangaroska, Sharma, Giannakos, Traetteberg, & Dillenbourg, 2018), mouse and keyboard pressure (Arapakis, Lalmas, & Valkansas, 2014;Begel, 2016), heart rate (Ahonen et al, 2016), and electro-dermal activity (Müller, 2015). Although data with additional features can help explain how students learn to program, to our knowledge, none of the hardware devices used to collect data from learners engaged in programming tasks is directly integrated with the IDE, which again imposes limitations on the range of data that can be collected.…”