Analyzing Gaze Behavior Using Object Detection and Unsupervised Clustering

Venuprasad, Pranav; Xu, Li; Huang, Enoch Hsin-Ho; Gilman, Andrew; Chukoskie, Leanne; Cosman, Pamela C.

doi:10.1145/3379155.3391316

Cited by 8 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Callemein et al [ 59 ] used measures for inter-rater agreement like Cohen’s

to show the performance of their gaze-to-face and gaze-to-hand mapping. Venuprasad et al [ 13 ] reported precision, recall, and accuracy for frames, and event metrics based on detection events: first looks, extra looks (i.e., revisits), false positive and false negative events are counted. Other works reported qualitative results only or did not evaluate their method.…”

Section: Discussionmentioning

confidence: 99%

“…They offer an interactive visualization for manual corrections. Venuprasad et al [ 13 ] use clustering with gaze and object locations from an object detection model to detect visual attention to an object or a face. Sümer et al [ 56 ] investigated the problem of automatic attention detection in a teaching scenario.…”

Section: Related Workmentioning

confidence: 99%

“…Previous methods tackled the automatic analysis of head-mounted eye tracking data in uninstrumented environments [ 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ]. Drawbacks of these methods include, e.g., a missing support for real-time applications, and the restriction to a limited number of classes (≤12).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Barz

Sonntag

2021

Sensors

View full text Add to dashboard Cite

Processing visual stimuli in a scene is essential for the human brain to make situation-aware decisions. These stimuli, which are prevalent subjects of diagnostic eye tracking studies, are commonly encoded as rectangular areas of interest (AOIs) per frame. Because it is a tedious manual annotation task, the automatic detection and annotation of visual attention to AOIs can accelerate and objectify eye tracking research, in particular for mobile eye tracking with egocentric video feeds. In this work, we implement two methods to automatically detect visual attention to AOIs using pre-trained deep learning models for image classification and object detection. Furthermore, we develop an evaluation framework based on the VISUS dataset and well-known performance metrics from the field of activity recognition. We systematically evaluate our methods within this framework, discuss potentials and limitations, and propose ways to improve the performance of future automatic visual attention detection methods.

show abstract

“…Callemein et al [ 59 ] used measures for inter-rater agreement like Cohen’s

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Barz

Sonntag

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Looking at the first direction, detecting viewers who are navigating in a similar way allows the quantitative assessment of user interactivity, and this can improve the accuracy and robustness of predictive algorithm but also allow the personalization of the streaming pipeline. Last, this quantitative assessment can play a key role also in healthcare applications, in which patients can be assessed based on their eye movement when consuming media content [ 189 ]. Looking at the user navigation as independent trajectories (e.g., tracking the center of the viewport displayed over time), users have been clustered via spectral clustering in [ 190 , 191 ].…”

Section: Learning-based Transmissionmentioning

confidence: 99%

Machine Learning for Multimedia Communications

Thomos

Maugey

Toni

2022

Sensors

View full text Add to dashboard Cite

Machine learning is revolutionizing the way multimedia information is processed and transmitted to users. After intensive and powerful training, some impressive efficiency/accuracy improvements have been made all over the transmission pipeline. For example, the high model capacity of the learning-based architectures enables us to accurately model the image and video behavior such that tremendous compression gains can be achieved. Similarly, error concealment, streaming strategy or even user perception modeling have widely benefited from the recent learning-oriented developments. However, learning-based algorithms often imply drastic changes to the way data are represented or consumed, meaning that the overall pipeline can be affected even though a subpart of it is optimized. In this paper, we review the recent major advances that have been proposed all across the transmission chain, and we discuss their potential impact and the research challenges that they raise.

show abstract

“…But these approaches come with certain limitations. Most of them rely on pre-trained computer vision models that do not allow for adapting the underlying model to a certain target domain [6,8,17,22,24,25]. These can be applied in very constrained settings only, i.e., if the dataset used for training the machine learning model matches the target domain.…”

Section: Introductionmentioning

confidence: 99%

Interactive Fixation-to-AOI Mapping for Mobile Eye Tracking Data based on Few-Shot Image Classification

Barz

Bhatti

Alam

et al. 2023

28th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Mobile eye tracking is an important tool in psychology and humancentred interaction design for understanding how people process visual scenes and user interfaces. However, analysing recordings from mobile eye trackers, which typically include an egocentric video of the scene and a gaze signal, is a time-consuming and largely manual process. To address this challenge, we propose a web-based annotation tool that leverages few-shot image classification and interactive machine learning (IML) to accelerate the annotation process. The tool allows users to efficiently map fixations to areas of interest (AOI) in a video-editing-style interface. It includes an IML component that generates suggestions and learns from user feedback using a few-shot image classification model initialised with a small number of images per AOI. Our goal is to improve the efficiency and accuracy of fixation-to-AOI mapping in mobile eye tracking.

show abstract

Analyzing Gaze Behavior Using Object Detection and Unsupervised Clustering

Cited by 8 publications

References 18 publications

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Machine Learning for Multimedia Communications

Interactive Fixation-to-AOI Mapping for Mobile Eye Tracking Data based on Few-Shot Image Classification

Contact Info

Product

Resources

About