Predicting task from eye movements: On the importance of spatial distribution, dynamics, and image features

Boisvert, Jonathan; Bruce, Neil D. B.

doi:10.1016/j.neucom.2016.05.047

Cited by 42 publications

(46 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the first eye-tracking experiments in history suggested that gaze patterns are strongly modulated by different task demands (Yarbus, 1965). This result has since been replicated and extended: More recent studies showed that the task at hand can even be inferred using gaze-based classifiers (Boisvert & Bruce, 2016;Borji & Itti, 2014;Haji-Abolhassani & Clark, 2014;Kanan et al, 2015). Here, gender appears to be the variable that produces the strongest differences between participants.…”

Section: Limitations Of This Studymentioning

confidence: 76%

Face exploration dynamics differentiate men and women

et al. 2016

View full text Add to dashboard Cite

The human face is central to our everyday social interactions. Recent studies have shown that while gazing at faces, each one of us has a particular eye-scanning pattern, highly stable across time. Although variables such as culture or personality have been shown to modulate gaze behavior, we still don't know what shapes these idiosyncrasies. Moreover, most previous observations rely on static analyses of small-sized eye-position data sets averaged across time. Here, we probe the temporal dynamics of gaze to explore what information can be extracted about the observers and what is being observed. Controlling for any stimuli effect, we demonstrate that among many individual characteristics, the gender of both the participant (gazer) and the person being observed (actor) are the factors that most influence gaze patterns during face exploration. We record and exploit the largest set of eye-tracking data (405 participants, 58 nationalities) from participants watching videos of another person. Using novel data-mining techniques, we show that female gazers follow a much more exploratory scanning strategy than males. Moreover, female gazers watching female actresses look more at the eye on the left side. These results have strong implications in every field using gaze-based models from computer vision to clinical psychology.

show abstract

Section: Limitations Of This Studymentioning

confidence: 76%

Face exploration dynamics differentiate men and women

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Inference from gaze data consists in deducing subjective characteristics solely from ocular data, such as age (Le Meur et al, 2017b), gender (Coutrot, Binetti, Harrison, Mareschal, & Johnston, 2016;Sammaknejad, Pouretemad, Eslahchi, Salahirad, & Alinejad, 2017), mental states and traits (Liao, Zhang, Zhu, & Ji, 2005;Hoppe, Loetscher, Morey, & Bulling, 2015;Yamada & Kobayashi, 2017;Hoppe, Loetscher, Morey, & Bulling, 2018), expertise and skill proficiency (Eivazi & Bednarik, 2011;Boccignone, Ferraro, Crespi, Robino, & de'Sperati, 2014;Tien et al, 2014;Kolodziej, Majkowski, Francuz, Rak, & Augustynowicz, 2018), and neurological disorders (Kupas, Harangi, Czifra, & Andrassy, 2017;Terao, Fukuda, & Hikosaka, 2017).It has proven useful in identifying autism spectrum disorder (Pierce et al, 2016), fetal alcohol spectrum disorder (Tseng, Paolozza, Munoz, Reynolds, & Itti, 2013), dementia (Zhang et al, 2016;Beltrán, García-Vázquez, Benois-Pineau, Gutierrez-Robledo, & Dartigues, 2018), dyslexia (Benfatto et al, 2016), anxiety (Abbott, Shirali, Haws, & Lack, 2017), mental fatigue (Yamada & Kobayashi, 2017), and other disorders. It has also been applied to task detection (Borji & Itti, 2014;Haji-Abolhassani & Clark, 2014;Kanan, Ray, Bseiso, Hsiao, & Cottrell, 2014;Boisvert & Bruce, 2016). In addition gaze is also utilized as a biometric clue (Holland & Komogortsev, 2011;Cantoni, Galdi, Nappi, Porta, & Riccio, 2015).…”

Section: Inference From Gaze Datamentioning

confidence: 99%

Predicting artificial visual field losses: A gaze-based inference study

et al. 2019

View full text Add to dashboard Cite

Visual field defects are a worldwide concern, and the proportion of the population experiencing vision loss is ever increasing. Macular degeneration and glaucoma are among the four leading causes of permanent vision loss. Identifying and characterizing visual field losses from gaze alone could prove crucial in the future for screening tests, rehabilitation therapies, and monitoring. In this experiment, 54 participants took part in a free-viewing task of visual scenes while experiencing artificial scotomas (central and peripheral) of varying radii in a gaze-contingent paradigm. We studied the importance of a set of gaze features as predictors to best differentiate between artificial scotoma conditions. Linear mixed models were utilized to measure differences between scotoma conditions. Correlation and factorial analyses revealed redundancies in our data. Finally, hidden Markov models and recurrent neural networks were implemented as classifiers in order to measure the predictive usefulness of gaze features. The results show separate saccade direction biases depending on scotoma type. We demonstrate that the saccade relative angle, amplitude, and peak velocity of saccades are the best features on the basis of which to distinguish between artificial scotomas in a free-viewing task. Finally, we discuss the usefulness of our protocol and analyses as a gaze-feature identifier tool that discriminates between artificial scotomas of different types and sizes.

show abstract

“…Since 2012, numerous studies have tried to classify observers’ gaze patterns according to the task at hand during reading (Henderson et al 2013 ), counting (Haji-Abolhassani and Clark 2013 ), searching (Zelinsky et al 2013 ), driving (Lemonnier et al 2014 ), mind wandering (Mills et al 2015 ), memorizing, and exploring static artificial or natural scenes (Kanan et al 2014 ; Borji and Itti 2014 ; Haji-Abolhassani and Clark 2014 ). For a thorough review of task-prediction algorithms, see (Boisvert and Bruce 2016 ). Eye movements can also be used to quantify mental workload , especially during demanding tasks such as air traffic control (Ahlstrom and Friedman-Berg 2006 ; Di Nocera et al 2006 ; Kang and Landry 2015 ; McClung and Kang 2016 ; Mannaru et al 2016 ).…”

Section: Introductionmentioning

confidence: 99%

Scanpath modeling and classification with hidden Markov models

2017

View full text Add to dashboard Cite

How people look at visual information reveals fundamental information about them; their interests and their states of mind. Previous studies showed that scanpath, i.e., the sequence of eye movements made by an observer exploring a visual stimulus, can be used to infer observer-related (e.g., task at hand) and stimuli-related (e.g., image semantic category) information. However, eye movements are complex signals and many of these studies rely on limited gaze descriptors and bespoke datasets. Here, we provide a turnkey method for scanpath modeling and classification. This method relies on variational hidden Markov models (HMMs) and discriminant analysis (DA). HMMs encapsulate the dynamic and individualistic dimensions of gaze behavior, allowing DA to capture systematic patterns diagnostic of a given class of observers and/or stimuli. We test our approach on two very different datasets. Firstly, we use fixations recorded while viewing 800 static natural scene images, and infer an observer-related characteristic: the task at hand. We achieve an average of 55.9% correct classification rate (chance = 33%). We show that correct classification rates positively correlate with the number of salient regions present in the stimuli. Secondly, we use eye positions recorded while viewing 15 conversational videos, and infer a stimulus-related characteristic: the presence or absence of original soundtrack. We achieve an average 81.2% correct classification rate (chance = 50%). HMMs allow to integrate bottom-up, top-down, and oculomotor influences into a single model of gaze behavior. This synergistic approach between behavior and machine learning will open new avenues for simple quantification of gazing behavior. We release SMAC with HMM, a Matlab toolbox freely available to the community under an open-source license agreement.

show abstract

Predicting task from eye movements: On the importance of spatial distribution, dynamics, and image features

Cited by 42 publications

References 52 publications

Face exploration dynamics differentiate men and women

Face exploration dynamics differentiate men and women

Predicting artificial visual field losses: A gaze-based inference study

Scanpath modeling and classification with hidden Markov models

Contact Info

Product

Resources

About