Predicting an observer's task using multi-fixation pattern analysis

Kanan, Christopher; Ray, Nicholas A.; Bseiso, Dina N. F.; Hsiao, Janet H.; Cottrell, Garrison W.

doi:10.1145/2578153.2578208

Cited by 45 publications

(41 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consistent with other studies (Borji & Itti, ;Kanan et al, 2014), we found that a subject's task can be inferred solely from their eye movements. Our algorithms were most successful when judging observer's tasks from their own history of eye movement patterns, from which we conclude that, at least for these tasks, observers have idiosyncratic scanpath routines.…”

Section: Resultssupporting

confidence: 92%

“…This approach is similar to the method used by Greene, Liu, and Wolfe (2012) and later by Kanan et al (2014). The summary statistics algorithm only captures the simplest temporal statistics, and it ignores all spatial properties and more complex temporal characteristics.…”

Section: Algorithmsmentioning

confidence: 99%

“…Multi-fixation pattern analysis (MFPA) is a new eye movement analysis technique that harnesses machine learning to make inferences about subjects from their eye movements (Benson et al, 2012;Greene, Liu, & Wolfe, 2012;Tseng et al, 2013;Kanan et al, 2014;Borji & Itti, 2014). MFPA algorithms take a person's scanpath, a sequence of fixations, as their input and use the scanpath to infer traits such as the task the person was given.…”

Section: Introductionmentioning

confidence: 99%

“…If an algorithm can make this inference above chance when trained on a person's scanpaths for specific tasks, then this suggests that the person might have scanpath routines for accomplishing one or more of the tasks. Prior work with MFPA has focused on validating the technique for inferring the task given to a subject when viewing scenes (Greene, Liu, & Wolfe, 2012;Kanan et al, 2014;Borji & Itti, 2014) and for inferring whether the subject has a particular disease (Benson et al, 2012;Tseng et al, 2013). In this paper, we use MFPA to determine if people have scanpath routines for making different inferences about faces.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Humans have idiosyncratic and task-specific scanpaths for judging faces

et al. 2015

Self Cite

View full text Add to dashboard Cite

a b s t r a c tSince Yarbus's seminal work, vision scientists have argued that our eye movement patterns differ depending upon our task. This has recently motivated the creation of multi-fixation pattern analysis algorithms that try to infer a person's task (or mental state) from their eye movements alone. Here, we introduce new algorithms for multi-fixation pattern analysis, and we use them to argue that people have scanpath routines for judging faces. We tested our methods on the eye movements of subjects as they made six distinct judgments about faces. We found that our algorithms could detect whether a participant is trying to distinguish angriness, happiness, trustworthiness, tiredness, attractiveness, or age. However, our algorithms were more accurate at inferring a subject's task when only trained on data from that subject than when trained on data gathered from other subjects, and we were able to infer the identity of our subjects using the same algorithms. These results suggest that (1) individuals have scanpath routines for judging faces, and that (2) these are diagnostic of that subject, but that (3) at least for the tasks we used, subjects do not converge on the same ''ideal'' scanpath pattern. Whether universal scanpath patterns exist for a task, we suggest, depends on the task's constraints and the level of expertise of the subject.

show abstract

Section: Resultssupporting

confidence: 92%

Section: Algorithmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Humans have idiosyncratic and task-specific scanpaths for judging faces

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…States of the model are typically represented by ROIs and state transitions are calculated as the transition probability between the different ROIs (Kanan et al, 2014).…”

Section: Probabilistic Scanpath Comparisonmentioning

confidence: 99%

SubsMatch 2.0: Scanpath comparison and classification based on subsequence frequencies

et al. 2016

View full text Add to dashboard Cite

Our eye movements are driven by a continuous trade-off between the need for detailed examination of objects of interest and the necessity to keep an overview of our surrounding. In consequence, behavioral patterns that are characteristic for our actions and their planning are typically manifested in the way we move our eyes to interact with our environment. Identifying such patterns from individual eye movement measurements is however highly challenging. In this work, we tackle the challenge of quantifying the influence of experimental factors on eye movement sequences. We introduce an algorithm for extracting sequence-sensitive features from eye movements and for the classification of eye movements based on the frequencies of small subsequences. Our approach is evaluated against the state-of-the art on a novel and a very rich collection of eye movements data derived from four experimental settings, from static viewing tasks to highly dynamic outdoor settings. Our results show that the proposed method is able to classify eye movement sequences over a variety of experimental designs. The choice of parameters is discussed in detail with special focus on highlighting different aspects of general scanpath shape. Algorithms and evaluation data are available at: www.ti.uni-tuebingen.de/scanpathcomparison.html.

show abstract