Characterizing joint attention behavior during real world interactions using automated object and gaze detection

Venuprasad, Pranav; Dobhal, Tushal; Paul, Anurag; Nguyen, Tu N. M.; Gilman, Andrew; Cosman, Pamela C.; Chukoskie, Leanne

doi:10.1145/3314111.3319843

Cited by 12 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a driving situation, realtime gaze coding allows to track a driver's attention and detecting whether they are distracted or fatigue [106,255], and support automated driving [170]. Gaze can also help diagnose mental health or autism by analysing the scan path [236]. Moving beyond, the gaze is also presented as a human-computer interface, facilitating controlling Internet of Things (IoT) devices in a smart home system [127], and empowering people with a physical impairment to interact with applications such as creative art [49,140,263].…”

Section: Introductionmentioning

confidence: 99%

An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices

Lei

Khamis

et al. 2023

ACM Comput. Surv.

View full text Add to dashboard Cite

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.

show abstract

Section: Introductionmentioning

confidence: 99%

An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices

Lei

Khamis

et al. 2023

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…In contrast to manual qualitative evaluation, merits of automated analysis include an unbiased judgement of the resultant gaze, enhanced by metrics to allow further prediction based on the level of attention. Frameworks for analysis of this calibre have been applied in a variety of domains ranging from social behaviour of adults [8,20,21] and infants [13], classroom environments [4], [3] to human-robot interaction [17] and industrial environments [6].…”

Section: Introductionmentioning

confidence: 99%

“…Lemaignan et al [12] studied one's focus time on a target, determined by a broad field of attention estimated by the head pose, limiting the accurate alignment of the gaze to a small dynamic target. A study of joint attention [20] estimates the latency between the instruction and the resulting look, based on the presence of the gaze dot within the expected field of attention, whilst the latency, longest and shortest look for each object of interest has been previously assessed [21]. However, these work [20,21] rely on specialised sensors to capture the gaze and object position, as opposed to a single video feed.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, the analysis of attention over a time frame in response to a dynamic physical target is a persisting research gap in the domain of attention analysis, given the invalidity of prior approaches primarily focusing on stationary targets [12,20,21]. As per the Diagnostic and Statistical Manual of Mental Disorders [2], children displaying prodromal symptoms of Autism Spectrum Disorder (ASD) show deficits in following another's pointing, eye gaze or movement of a target object.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Automated Temporal Analysis of Gaze Following in a Visual Tracking Task

Dhanawansa

Samarasinghe

Gardiner

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The attention assessment of an individual in following the motion of a target object provides valuable insights into understanding one's behavioural patterns in cognitive disorders including Autism Spectrum Disorder (ASD). Existing frameworks often require dedicated devices for gaze capture, focus on stationary target objects, or restrict the analysis to a single image obstructing a temporal analysis of the participant's response. This drives the motivation to address the persisting research gap in the analysis of the video capture of a visual tracking task of a dynamic physical object, over a time frame. To this end, this paper proposes a novel framework to analyse the temporal relationship between the 3D head pose angles and object displacement, and demonstrates its validity via application on the EYEDIAP video dataset. The conducted multivariate time-series analysis is two-fold; the statistical correlation computes the similarity between the time series as an overall measure of attention; and the Dynamic Time Warping (DTW) algorithm aligns the sequences of the target trajectory and the gaze, and computes relevant temporal metrics. It was proven that the correlation and metrics relevant to DTW corroborated each other. Additionally, the latency and maximum time of focus retention extracted as temporal characteristics, enabled an intragroup comparison between the performance of the participants. An extension to the analysis of the behavioural response concluded that the response to horizontal motion generally outperformed that of vertical motion, and that 56.3% of participants improved their retention of focus on the vertical motion of the target over time, implying that following a vertical target initially proved a challenging task.

show abstract

“…Public perception of these devices is overwhelmingly negative, as seen with the initial release of the Google Glass, as they infringe on the privacy of both the user and bystanders [20,59,70]. Daily users of eye tracking technology trade-off the privacy of their everyday actions for the benefit of activity logging, gaze-based interfaces, and assistive applications [7,37,55,81]. Steil et al have developed a privacy approach specifically for the scene camera, using a controlled shutter to disable the video feed in private situations [75].…”

Section: Introductionmentioning

confidence: 99%

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

John

Jörg

Koppal

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

The gaze behavior of virtual avatars is critical to social presence and perceived eye contact during social interactions in Virtual Reality. Virtual Reality headsets are being designed with integrated eye tracking to enable compelling virtual social interactions. This paper shows that the near infra-red cameras used in eye tracking capture eye images that contain iris patterns of the user. Because iris patterns are a gold standard biometric, the current technology places the user's biometric identity at risk. Our first contribution is an optical defocus based hardware solution to remove the iris biometric from the stream of eye tracking images. We characterize the performance of this solution with different internal parameters. Our second contribution is a psychophysical experiment with a same-different task that investigates the sensitivity of users to a virtual avatar's eye movements when this solution is applied. By deriving detection threshold values, our findings provide a range of defocus parameters where the change in eye movements would go unnoticed in a conversational setting. Our third contribution is a perceptual study to determine the impact of defocus parameters on the perceived eye contact, attentiveness, naturalness, and truthfulness of the avatar. Thus, if a user wishes to protect their iris biometric, our approach provides a solution that balances biometric protection while preventing their conversation partner from perceiving a difference in the user's virtual avatar. This work is the first to develop secure eye tracking configurations for VR/AR/XR applications and motivates future work in the area.

show abstract

Characterizing joint attention behavior during real world interactions using automated object and gaze detection

Cited by 12 publications

References 12 publications

An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices

An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices

The Automated Temporal Analysis of Gaze Following in a Visual Tracking Task

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

Contact Info

Product

Resources

About