The right look for the job: decoding cognitive processes involved in the task from spatial eye-movement patterns

Król, Magdalena Ewa; Król, Michał

doi:10.1007/s00426-018-0996-5

Cited by 8 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While this manipulation is reminiscent of other experiments relying on the bottom-up influence of words and pictures (e.g., Henderson et al, 2013;Boisvert & Bruce, 2016) the eye movements in the Coco and Keller (2014) tasks can be attributed to the occurrence of top-down attentional processes. A conceptually related follow-up to this study classified tasks along two spatial and semantic dimensions, resulting in 51% classification accuracy (chance = 25%; Król & Król, 2018). A closer look at these results showed that the categories within the semantic dimension were consistently misclassified, suggesting that this level of distinction may require a richer dataset, or a more powerful decoding algorithm.…”

Section: Introductionmentioning

confidence: 68%

“…Eye movements can only delineate tasks to the extent that the cognitive processes underlying the tasks can be differentiated (Król & Król, 2018). Every task is associated with a unique set of cognitive processes (Coco & Keller, 2014;Król & Król, 2018), but in some cases, the cognitive processes for different tasks may produce indistinguishable eye movement patterns. To differentiate the cognitive processes underlying task-evoked eye movements, some studies have chosen to classify tasks that rely on stimuli that prompt easily distinguishable eye movements, such as reading text (e.g., Henderson, Shinkareva, Wang, Luke, & Olejarczyk, 2013).…”

Section: Introductionmentioning

confidence: 99%

“…For example, MacInnes et al (2018) attempted to provide an algorithm with data designed to be representative of inputs to the frontal eye fields. In some instances, such as the case of Król and Król (2018), grounding the data using theoretically driven aggregation methods may require sacrificing granularity in the dataset. This means that aggregating the data has the potential to wash out certain fine-grained distinctions that could otherwise be detected.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Convolutional neural networks can decode eye movement data: A black box approach to predicting task from eye movements

Cole¹,

Kuntzelman²,

Dodd³

et al. 2020

Preprint

View full text Add to dashboard Cite

Previous attempts to classify task from eye movement data have relied on model architectures designed to emulate theoretically defined cognitive processes, and/or data that has been processed into aggregate (e.g., fixations, saccades) or statistical (e.g., fixation density) features. _Black box_ convolutional neural networks (CNNs) are capable of identifying relevant features in raw and minimally processed data and images, but difficulty interpreting these model architectures has contributed to challenges in generalizing lab-trained CNNs to applied contexts. In the current study, a CNN classifier was used to classify task from two eye movement datasets (Exploratory and Confirmatory) in which participants searched, memorized, or rated indoor and outdoor scene images. The Exploratory dataset was used to tune the hyperparameters of the model, and the resulting model architecture was re-trained, validated, and tested on the Confirmatory dataset. The data were formatted into timelines (i.e., x-coordinate, y-coordinate, pupil size) and minimally processed images. To further understand the informational value of each component of the eye movement data, the timeline and image datasets were broken down into subsets with one or more components systematically removed. Classification of the timeline data consistently outperformed the image data. The Memorize condition was most often confused with Search and Rate. Pupil size was the least uniquely informative component when compared with the x- and y-coordinates. The general pattern of results for the Exploratory dataset was replicated in the Confirmatory dataset. Overall, the present study provides a practical and reliable black box solution to classifying task from eye movement data.

show abstract

Section: Introductionmentioning

confidence: 68%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convolutional neural networks can decode eye movement data: A black box approach to predicting task from eye movements

Cole¹,

Kuntzelman²,

Dodd³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Eye-tracking technology has been used in studying cognitive states during the problem-solving process in many previous works, including [12], IQ [13], cognitive states [9,14,15], visual perception [16], and visual cognition [17]. Beyond learning science, eye-tracking technology also performs well in safe driving assessment [18], diagnosis of Alzheimer's disease [19], recognizing stimuli types [20,21], and gaming [22]. These studies concluded that a lot of eye-tracking characteristics, such as fixation duration [23], saccades, path distance, blink rate, and pupil diameter, are considered to reflect important information of the cognitive process.…”

Section: Related Workmentioning

confidence: 99%

“…The information of both the stimuli and the problem solvers could be reflected in the eye-tracking data. The types of stimuli have been successfully predicted by using eye-tracking data in reading materials types [21] and landscape types [20]. In addition, several former works investigated the eye-movement patterns in the problem-solving process and connected them to other information related to the participants.…”

Section: Related Workmentioning

confidence: 99%

Predicting Spatial Visualization Problems’ Difficulty Level from Eye-Tracking Data

Younes

Bairaktarova

et al. 2020

Sensors

View full text Add to dashboard Cite

The difficulty level of learning tasks is a concern that often needs to be considered in the teaching process. Teachers usually dynamically adjust the difficulty of exercises according to the prior knowledge and abilities of students to achieve better teaching results. In e-learning, because there is no teacher involvement, it often happens that the difficulty of the tasks is beyond the ability of the students. In attempts to solve this problem, several researchers investigated the problem-solving process by using eye-tracking data. However, although most e-learning exercises use the form of filling in blanks and choosing questions, in previous works, research focused on building cognitive models from eye-tracking data collected from flexible problem forms, which may lead to impractical results. In this paper, we build models to predict the difficulty level of spatial visualization problems from eye-tracking data collected from multiple-choice questions. We use eye-tracking and machine learning to investigate (1) the difference of eye movement among questions from different difficulty levels and (2) the possibility of predicting the difficulty level of problems from eye-tracking data. Our models resulted in an average accuracy of 87.60% on eye-tracking data of questions that the classifier has seen before and an average of 72.87% on questions that the classifier has not yet seen. The results confirmed that eye movement, especially fixation duration, contains essential information on the difficulty of the questions and it is sufficient to build machine-learning-based models to predict difficulty level.

show abstract

Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data

Castilla

Catala

Pons

et al. 2023

User Model User-Adap Inter

View full text Add to dashboard Cite

Eye-tracking techniques are widely used to analyze user behavior. While eye-trackers collect valuable quantitative data, the results are often described in a qualitative manner due to the lack of a model that interprets the gaze trajectories generated by routine tasks, such as reading or comparing two products. The aim of this work is to propose a new quantitative way to analyze gaze trajectories (scanpaths) using machine learning. We conducted a within-subjects study (N = 30) testing six different tasks that simulated specific user behaviors in web sites (attentional, comparing two images, reading in different contexts, and free surfing). We evaluated the scanpath results with three different classifiers (long short-term memory recurrent neural network—LSTM, random forest, and multilayer perceptron neural network—MLP) to discriminate between tasks. The results revealed that it is possible to classify and distinguish between the 6 different web behaviors proposed in this study based on the user’s scanpath. The classifier that achieved the best results was the LSTM, with a 95.7% accuracy. To the best of our knowledge, this is the first study to provide insight about MLP and LSTM classifiers to discriminate between tasks. In the discussion, we propose practical implications of the study results.

show abstract

The right look for the job: decoding cognitive processes involved in the task from spatial eye-movement patterns

Cited by 8 publications

References 32 publications

Convolutional neural networks can decode eye movement data: A black box approach to predicting task from eye movements

Convolutional neural networks can decode eye movement data: A black box approach to predicting task from eye movements

Predicting Spatial Visualization Problems’ Difficulty Level from Eye-Tracking Data

Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data

Contact Info

Product

Resources

About