Assessing Emotion by Mouse-cursor Tracking: Theoretical and Empirical Rationales

Yamauchi, Takashi; Leontyev, Anton; Razavi, Moein

doi:10.1109/acii.2019.8925537

Cited by 14 publications

(8 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though reaction time has traditionally been used to measure cognitive conflict (Shtulman & Valcarcel, 2012), recent work suggests mouse-tracking provides a more sensitive measure of ongoing conflict than reaction time (Yamauchi et al, 2019). This study used maximum deviation as a measure of implicit processing.…”

Section: Methodsmentioning

confidence: 99%

Contextualized knowledge reduces misconceived COVID-19 health decisions.

Murray¹,

Willer²,

Arner³

et al. 2021

Journal of Applied Research in Memory and Cognition

View full text Add to dashboard Cite

How do we resolve conflicting ideas about how to protect our health during a pandemic? Prior knowledge influences our decisions, potentially creating implicit cognitive conflict with new, correct information. COVID-19 provides a natural condition for investigating how an individual's health-specific knowledge (e.g., understanding mask efficacy) and their personal context (e.g., outbreak proximity) influence their protective health behavior endorsement, as information about the virus, its spread, and lethality has changed over time. Using a dual-process-model framework, we investigated the role cognitive conflict has on health decision-making. We used a computer mousetracking paradigm alongside geographical information systems (GIS) as a proxy for context. The results support a contextualized-deficit-model framework in which relevant knowledge and context-based factors help individuals override cognitive conflict to make more preventative health decisions. Findings from this study may provide evidence for a more effective way for experts to combat non-adherence due to conflicting health information.

show abstract

Section: Methodsmentioning

confidence: 99%

Contextualized knowledge reduces misconceived COVID-19 health decisions.

Murray¹,

Willer²,

Arner³

et al. 2021

Journal of Applied Research in Memory and Cognition

View full text Add to dashboard Cite

show abstract

“…The results revealed that mouse usage (operationalized as the distance from an ideal line and the number of directional changes) was correlated to some of the emotional states measured in each experiment, but the correlations were not entirely consistent across the experiments. In two similar experiments (total N = 355), Yamauchi et al (2019) demonstrated that viewing emotional pictures affected mouse usage (measured as the distance from an ideal line and the area under the curve as spatial mouse features and the peak velocity and acceleration of the mouse as temporal features). Pimenta et al (2016) collected ten different mouse usage features (e.g., speed, acceleration, time between two mouse clicks, distance between two mouse clicks) of 24 participants during classwork in a computer laboratory to predict the self-rated fatigue level at 81% accuracy.…”

Section: The Computer Mouse As a Stress Detectormentioning

confidence: 96%

Tracking stress via the computer mouse? Promises and challenges of a potential behavioral stress marker

et al. 2021

View full text Add to dashboard Cite

Computer mouse tracking offers a simple and cost-efficient way to gather continuous behavioral data and has mostly been utilized in psychological science to study cognitive processes. The present study extends the potential applicability of computer mouse tracking and investigates the feasibility of using computer mouse tracking for stress measurement. Drawing on first empirical results and theoretical considerations, we hypothesized that stress affects sensorimotor processes involved in mouse usage. To explore the relationship between stress and computer mouse usage, we conducted a between-participant field experiment in which N = 994 participants worked on four mouse tasks in a high-stress or low-stress condition. In the manipulation check, participants reported different stress levels between the two conditions. However, frequentist and machine learning data analysis approaches did not reveal a clear and systematic relationship between mouse usage and stress. These findings challenge the feasibility of using straightforward computer mouse tracking for generalized stress measurement.

show abstract

“…Leontyev et al combined user response time and mouse movement features with machine learning technics and found an improvement in the accuracy of predicting attention-deficit/hyperactivity disorder (ADHD) [9][10][11]. Yamauchi et al combined behavioral measures and multiple mouse motion features to better predict people's emotions and cognitive conflict in computer tasks [12][13][14][15].…”

Section: Related Workmentioning

confidence: 99%

Multimodal-Multisensory Experiments

Razavi¹,

Yamauchi²,

Janfaza³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior—task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals—gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion—with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

show abstract

Assessing Emotion by Mouse-cursor Tracking: Theoretical and Empirical Rationales

Cited by 14 publications

References 52 publications

Contextualized knowledge reduces misconceived COVID-19 health decisions.

Contextualized knowledge reduces misconceived COVID-19 health decisions.

Tracking stress via the computer mouse? Promises and challenges of a potential behavioral stress marker

Multimodal-Multisensory Experiments

Contact Info

Product

Resources

About