Onfocus detection: identifying individual-camera eye contact from unconstrained images

Abstract

Onfocus detection aims at identifying whether the focus of the individual captured by a camera is on the camera or not. Based on the behavioral research, the focus of an individual during face-to-camera communication leads to a special type of eye contact, i.e., the individual-camera eye contact, which is a powerful signal in social communication and plays a crucial role in recognizing irregular individual status (e.g., lying or suffering mental disease) and special purposes (e.g., seeking help or attracting fans). Thus, developing effective onfocus detection algorithms is of significance for assisting the criminal investigation, disease discovery, and social behavior analysis. However, the review of the literature shows that very few efforts have been made toward the development of onfocus detector owing to the lack of large-scale public available datasets as well as the challenging nature of this task. To this end, this paper engages in the onfocus detection research by addressing the above two issues. Firstly, we build a large-scale onfocus detection dataset, named as the onfocus detection in the wild (OFDIW). It consists of 20623 images in unconstrained capture conditions (thus called “in the wild”) and contains individuals with diverse emotions, ages, facial characteristics, and rich interactions with surrounding objects and background scenes. On top of that, we propose a novel end-to-end deep model, i.e., the eye-context interaction inferring network (ECIIN), for onfocus detection, which explores eye-context interaction via dynamic capsule routing. Finally, comprehensive experiments are conducted on the proposed OFDIW dataset to benchmark the existing learning models and demonstrate the effectiveness of the proposed ECIIN.

Capsule Networks With Residual Pose Routing

“…3) Routing layers: To study the performance of routing layers in one block, we compare our ResCaps and one modified versions, i.e., ResCaps-3L. Specifically, ResCaps and ResCaps-3L consist of two and three ResP layer(s) 4 in one block, respectively. From Table V, we find that our ResCaps consisting of two ResP layers can achieve promising performance, compared to ResCaps-3L that employs three ResP layers in one block.…”

“…They can recognize the image by detecting the existence of a specific entity, i.e., invariance. However, an unsophisticated perturbation on the image can fool a well-trained network to fail in recognition [1]- [4]. More worryingly, natural and non-adversarial pose changes of familiar objects in the real world are enough to trick deep networks [5], [6].…”

SA-BiGCN: Bi-Stream Graph Convolution Networks With Spatial Attentions for the Eye Contact Detection in the Wild

“…detection becomes increasingly important in understanding the intention of surrounding pedestrians in the autonomous driving environment [5], [6], [7]. Many studies on eyes contact detection use vision images taken close to a person with a clear facial appearance [8], [9], [10], [11], [12]. However, pedestrian eye detection in the wild uses distant images or videos from vehicle sensors which brings great challenges to the problem given the image quality, unconstrained surroundings and illuminations [6], [7].…”

Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics

“…However, they usually assume that the training and testing data is balanced. In practice, training or testing data appears to be long-tailed, e.g., there exist few samples for rare diseases in medical diagnosis [19,35,36,39,42,45] or endangered animals in species classification [5,31,37]. As mentioned by [32], the case becomes even worse in weakly and semi-supervised learning scenarios [10,20,27,31,33,34,38,43,44].…”