Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios

The human eye gaze plays a vital role in monitoring people’s attention, and various efforts have been made to improve in-vehicle driver gaze tracking systems. Most of them build the specific gaze estimation model by pre-annotated data training in an offline way. These systems usually tend to have poor generalization performance during the online gaze prediction, which is caused by the estimation bias between the training domain and the deployment domain, making the predicted gaze points shift from their correct location. To solve this problem, a novel driver’s eye gaze tracking method with non-linear gaze point refinement is proposed in a monitoring system using two cameras, which eliminates the estimation bias and implicitly fine-tunes the gaze points. Supported by the two-stage gaze point clustering algorithm, the non-linear gaze point refinement method can gradually extract the representative gaze points of the forward and mirror gaze zone and establish the non-linear gaze point re-mapping relationship. In addition, the Unscented Kalman filter is utilized to track the driver’s continuous status features. Experimental results show that the non-linear gaze point refinement method outperforms several previous gaze calibration and gaze mapping methods, and improves the gaze estimation accuracy even on the cross-subject evaluation. The system can be used for predicting the driver’s attention.

Section: Experiments and Discussionmentioning

confidence: 99%

Dual-Cameras-Based Driver’s Eye Gaze Tracking System with Non-Linear Gaze Point Refinement

Ding

Yuan

2022

“…We propose a cheap and non-intrusive gaze focalization method based on a desktop static camera focused on the driver, the OpenFace 2.0 tool [6], and a calibration method using the NARMAX algorithm [35] to evaluate the driver's gaze. This proposal has been validated in a challenging accidental scenario (DADA2000 [36]) and published by the authors in [7]. The method is satisfactory enough to be an alternative to active desktop-mounted eye trackers or head-mounted eye trackers, which are intrusive, costly and difficult to utilise in physical environments.…”

Section: Driver's Gaze Focalization (Subsystem 1)mentioning

confidence: 97%

“…This data is unique for each user because the model fits the user under test from scratch. The driver must perform this task as naturally as possible, achieving a model that estimates gaze focalization at 30 fps, the best frame rate according to our ablation study presented on [7]. In total, 1200 samples are obtained per user to train the model.…”

Section: Driver's Gaze Focalization (Subsystem 1)mentioning

confidence: 99%

“…The main contributions to the take-over request study of this paper are: (1) the gaze focalization method based on a camera, which uses OpenFace 2.0 toolkit [6] and our own NARMAX calibration technique [7], and its fusion with a semantic segmentation road image to detect the focalized objects; (2) the development of a challenging take-over urban scenario based on CARLA consisting of transferring the control in an approaching manoeuvre with a vehicle ahead as a function of the TTC; (3) the calculation of a unified framework based on gaze focalization and vehicle data to evaluate driver's take-over performance in real-time; and (4) the implementation of a novel DSA metric using semantic segmentation and gaze tracking to assess driver's awareness.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Driver Take-Over Behaviour Study Based on Gaze Focalization and Vehicle Data in CARLA Simulator

Araluce

Bergasa

Ocaña

et al. 2022

Self Cite

Autonomous vehicles are the near future of the automobile industry. However, until they reach Level 5, humans and cars will share this intermediate future. Therefore, studying the transition between autonomous and manual modes is a fascinating topic. Automated vehicles may still need to occasionally hand the control to drivers due to technology limitations and legal requirements. This paper presents a study of driver behaviour in the transition between autonomous and manual modes using a CARLA simulator. To our knowledge, this is the first take-over study with transitions conducted on this simulator. For this purpose, we obtain driver gaze focalization and fuse it with the road’s semantic segmentation to track to where and when the user is paying attention, besides the actuators’ reaction-time measurements provided in the literature. To track gaze focalization in a non-intrusive and inexpensive way, we use a method based on a camera developed in previous works. We devised it with the OpenFace 2.0 toolkit and a NARMAX calibration method. It transforms the face parameters extracted by the toolkit into the point where the user is looking on the simulator scene. The study was carried out by different users using our simulator, which is composed of three screens, a steering wheel and pedals. We distributed this proposal in two different computer systems due to the computational cost of the simulator based on the CARLA simulator. The robot operating system (ROS) framework is in charge of the communication of both systems to provide portability and flexibility to the proposal. Results of the transition analysis are provided using state-of-the-art metrics and a novel driver situation-awareness metric for 20 users in two different scenarios.

“…Ledezma et al [42] also explored a feature-based method to predict the driver's gaze location. Araluce et al [43] located eye gaze via opensource toolkit in accidental scenarios. Chiou et al [44] monitored the driver eye gaze using sparse representation with part-based temporal face descriptors.…”

Section: Driver's Gaze Zone Estimationmentioning

confidence: 99%

Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion

Yuan

2022

Head pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification.