Driver Gaze Estimation in the Real World: Overcoming the Eyeglass Challenge

Rangesh, Akshay; Zhang, Bowen; Trivedi, Mohan M.

doi:10.1109/iv47402.2020.9304573

Cited by 40 publications

(29 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kim et al [52] utilize a GAN to convert low-light eye images into bright eye images. Rangesh et al [53] use a GAN to remove eyeglasses.…”

Section: A Deep Feature From Appearancementioning

confidence: 99%

“…Eye image [26] - [42], [119], [113] [38], [47], [48], [95], [120] [40], [46], [51], [98], [100], [115], [121] [49], [50], [79], [52], [53], [80], [54] -Facial image - [35] [43], [64], [101] [66], [68], [104], [108] [5], [46], [69], [92], [99], [110], [111], [116], [118], [122] [49], [50], [67], [59], [61], [83], [62], [123], [78], [124], [63], [112], [82], [37], [107] [60]…”

Section: Featurementioning

confidence: 99%

“…[43], [64], [42], [101], [113], [119] [38], [47], [66], [68], [95], [104], [108], [120] [5], [36], [40], [46], [70], [71], [72], [98], [99], [100], [115], [116], [118], [121], [122] [82], [50], [49], [37], [59], [61], [83], [62], [123], [78], [79], [124], [52], [53], [80], [63], [125], [107] -Semi-/Self-/Un-Supervised CNN --- [48] [51], [69], [92], [110], [111] [54], [112], [37...…”

Section: Modelmentioning

confidence: 99%

“…Single camera [26] [35] [43], [64], [42], [113] [38], [47], [48], [66], [68], [95], [104], [108], [120] [5], [36], [40], [46], [69], [70], [71], [72], [100], [111], [115], [116], [118], [121], [122] [59], [49], [50], [61], [83], [62], [123], [78], [79], [124], [52], [53], [80], [63], [125], [82], [37], [107] [60]…”

Section: Cameramentioning

confidence: 99%

See 3 more Smart Citations

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

View full text Add to dashboard Cite

Gaze estimation reveals where a person is looking. It is an important clue for understanding human intention. The recent development of deep learning has revolutionized many computer vision tasks, the appearance-based gaze estimation is no exception. However, it lacks a guideline for designing deep learning algorithms for gaze estimation tasks. In this paper, we present a comprehensive review of the appearance-based gaze estimation methods with deep learning. We summarize the processing pipeline and discuss these methods from four perspectives: deep feature extraction, deep neural network architecture design, personal calibration as well as device and platform. Since the data pre-processing and post-processing methods are crucial for gaze estimation, we also survey face/eye detection method, data rectification method, 2D/3D gaze conversion method, and gaze origin conversion method. To fairly compare the performance of various gaze estimation approaches, we characterize all the publicly available gaze estimation datasets and collect the code of typical gaze estimation algorithms. We implement these codes and set up a benchmark of converting the results of different methods into the same evaluation metrics. This paper not only serves as a reference to develop deep learning-based gaze estimation methods but also a guideline for future gaze estimation research. Implemented methods and data processing codes are available at http://phi-ai.org/GazeHub.

show abstract

“…Kim et al [52] utilize a GAN to convert low-light eye images into bright eye images. Rangesh et al [53] use a GAN to remove eyeglasses.…”

Section: A Deep Feature From Appearancementioning

confidence: 99%

Section: Featurementioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

Section: Cameramentioning

confidence: 99%

See 2 more Smart Citations

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A large body of literature has addressed driver behavior analysis using in-cabin vision sensors. The most commonly addressed task is driver gaze estimation [5]- [16], since the driver's gaze closely relates to their attention to driving and non-driving tasks. Early works relied on head pose estimation [5], [6], [8], [17] or a combination of head and eye features [7], [9]- [11], [18] for estimating the driver's gaze.…”

Section: A Vision Based Driver Behavior Analysismentioning

confidence: 99%

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Rangesh¹,

Deo²,

Greer³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Understanding occupant-vehicle interactions by modeling control transitions is important to ensure safe approaches to passenger vehicle automation. Models which contain contextual, semantically meaningful representations of driver states can be used to determine the appropriate timing and conditions for transfer of control between driver and vehicle. However, such models rely on real-world control take-over data from drivers engaged in distracting activities, which is costly to collect. Here, we introduce a scheme for data augmentation for such a dataset. Using the augmented dataset, we develop and train take-over time (TOT) models that operate sequentially on mid and high-level features produced by computer vision algorithms operating on different driver-facing camera views, showing models trained on the augmented dataset to outperform the initial dataset. The demonstrated model features encode different aspects of the driver state, pertaining to the face, hands, foot and upper body of the driver. We perform ablative experiments on feature combinations as well as model architectures, showing that a TOT model supported by augmented data can be used to produce continuous estimates of take-over times without delay, suitable for complex real-world scenarios.

show abstract