LAEO-Net++: Revisiting People Looking at Each Other in Videos

Marín-Jiménez, Manuel J.; Kalogeiton, Vicky; Medina-Suarez, Pablo; Zisserman, Andrew

doi:10.1109/tpami.2020.3048482

Cited by 10 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computer vision community has a long history of works on perceiving TV shows/movies. One of the most common tasks in this setting is studying the show characters with emphasis in face/character identification [2,9,39,43,57,58], where different cues have also been explored, e.g., body, voice or gaze [6,8,34,35].…”

Section: Perceiving Tv Showsmentioning

confidence: 99%

“…Ferrari et al study 2D pose estimation [10] and perform pose-based analysis [11]. Patron et al [44] and Hoai et al [19] focus on human interactions, while Recasens et al [46] and Marín-Jiménez et al [34] use this data to study gaze. Vondrick et al [60] use sequences from TV shows to learn activity forecasting, while Wang et al [61] leverage it for affordance learning.…”

Section: Perceiving Tv Showsmentioning

confidence: 99%

See 1 more Smart Citation

The One Where They Reconstructed 3D Humans and Environments in TV Shows

Pavlakos¹,

Weber²,

Tancik³

et al. 2022

Preprint

View full text Add to dashboard Cite

that can guide and improve the recovery of 3D human pose and position in these environments. Moreover, we show that reasoning about humans and their environment in 3D enables a broad range of downstream applications: re-identification, gaze estimation, cinematography and image editing. We apply our approach on environments from seven iconic TV shows and perform an extensive evaluation of the proposed system.

show abstract

Section: Perceiving Tv Showsmentioning

confidence: 99%

Section: Perceiving Tv Showsmentioning

confidence: 99%

The One Where They Reconstructed 3D Humans and Environments in TV Shows

Pavlakos¹,

Weber²,

Tancik³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For the purpose of human communication analysis [e.g., detection of Autism Spectrum Disorder (ASD) or social interactions], several computer-vision-based methods have been developed for detecting eye contacts (mutual-gaze) ( Marin-Jimenez et al, 2019 ) or joint attention ( Recasens et al, 2015 ). These method uses third-person-video where a filmer took a video.…”

Section: Technical Challenges Toward Artificial Systems That Incorporate Humanitude Techniquesmentioning

confidence: 99%

Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™

et al. 2021

View full text Add to dashboard Cite

Due to cognitive and socio-emotional decline and mental diseases, senior citizens, especially people with dementia (PwD), struggle to interact smoothly with their caregivers. Therefore, various care techniques have been proposed to develop good relationships with seniors. Among them, Humanitude is one promising technique that provides caregivers with useful interaction skills to improve their relationships with PwD, from four perspectives: face-to-face interaction, verbal communication, touch interaction, and helping care receivers stand up (physical interaction). Regardless of advances in elderly care techniques, since current social robots interact with seniors in the same manner as they do with younger adults, they lack several important functions. For example, Humanitude emphasizes the importance of interaction at a relatively intimate distance to facilitate communication with seniors. Unfortunately, few studies have developed an interaction model for clinical care communication. In this paper, we discuss the current challenges to develop a social robot that can smoothly interact with PwDs and overview the interaction skills used in Humanitude as well as the existing technologies.

show abstract

“…Recently, (Palmero et al 2018) built a mutual gaze detection model on top of 3D gaze vectors estimated using a CNN and the locations of faces in the image. Instead of explicitly estimating the 3D gaze and inferring mutual gaze from it, (Marin-Jimenez et al 2019) proposed LAEO-Net for directly detecting mutual gaze in videos. LAEO-Net consists of two CNNs for encoding head images and head location map, respectively.…”

Section: Mutual Gaze Detectionmentioning

confidence: 99%

“…Among all verbal and non-verbal cues, gaze is known to be one of the most informative cues about social interactions. Existing works have shown that gaze can be used as a reliable proxy for identifying the social relationships between people in a scene (Fan et al 2018(Fan et al , 2019Marin-Jimenez et al 2019). In particular, mutual eye contact between two people in an image often indicates social connectivity, friendliness and intimacy (Abele 1986).…”

Section: Introductionmentioning

confidence: 99%

Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze

Doosti

Chen

Vemulapalli

et al. 2021

AAAI

View full text Add to dashboard Cite

Mutual gaze detection, i.e., predicting whether or not two people are looking at each other, plays an important role in understanding human interactions. In this work, we focus on the task of image-based mutual gaze detection, and propose a simple and effective approach to boost the performance by using an auxiliary 3D gaze estimation task during the training phase. We achieve the performance boost without additional labeling cost by training the 3D gaze estimation branch using pseudo 3D gaze labels deduced from mutual gaze labels. By sharing the head image encoder between the 3D gaze estimation and the mutual gaze detection branches, we achieve better head features than learned by training the mutual gaze detection branch alone. Experimental results on three image datasets show that the proposed approach improves the detection performance significantly without additional annotations. This work also introduces a new image dataset that consists of 33.1K pairs of humans annotated with mutual gaze labels in 29.2K images.

show abstract

LAEO-Net++: Revisiting People Looking at Each Other in Videos

Cited by 10 publications

References 39 publications

The One Where They Reconstructed 3D Humans and Environments in TV Shows

The One Where They Reconstructed 3D Humans and Environments in TV Shows

Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™

Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze

Contact Info

Product

Resources

About