AutoPOSE: Large-scale Automotive Driver Head Pose and Gaze Dataset with Deep Head Orientation Baseline

Selim, Mohamed; Firintepe, Ahmet; Pagani, Alain; Stricker, Didier

doi:10.5220/0009330105990606

Cited by 23 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To compare our two methods P2R and P2P with the two selected state-of-the-art approaches, we define the following metrics for further benchmarking of our results: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and Balanced Mean Angular Error (BMAE). The BMAE considers the unbalanced amount of different head orientations by introducing section definition [ 38 , 39 , 40 ]:

…”

Section: Discussionmentioning

confidence: 99%

From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

et al. 2021

Self Cite

View full text Add to dashboard Cite

In this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid model combining Deep Learning and a voting-based mechanism. Our methods utilize a point cloud estimator, which we trained on multi-view infrared images in a semi-supervised manner, generating point clouds based on one image only. We generate a point cloud dataset with our point cloud estimator using the HMDPose dataset, consisting of multi-view infrared images of various AR glasses with the corresponding 6-DoF poses. In comparison to another point cloud-based 6-DoF pose estimation named CloudPose, we achieve an error reduction of around 50%. Compared to a state-of-the-art image-based method, we reduce the pose estimation error by around 96%.

show abstract

…”

Section: Discussionmentioning

confidence: 99%

From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Databases focusing on head pose estimation aim to provide frame-based annotations for the driver's head pose [27]- [32], which has a wide range of applications including coarse gaze estimation, driver behavior modeling, and human-computer interaction (HCI) for entertainment purposes. Table I summarizes the main databases created to estimate head pose of the driver.…”

Section: A Databases Focusing On Head Pose Estimationmentioning

confidence: 99%

“…In total, the dataset has 250k frames. Another dataset collected in a car simulator is the AutoPose corpus [32]. This corpus relies on one dashboard IR camera running at 60 fps, and a Kinect V2 camera (RGB, Depth, and Infrared), mounted at the rear view mirror, running at 30 fps.…”

Section: A Databases Focusing On Head Pose Estimationmentioning

confidence: 99%

“…The reliability depends on the number of markers, the frame rate and resolution of the camera, and environmental factors such as background illumination and occlusion. A Mo-Cap system requires a sophisticated setup, so it is more appropriate for data collections in car-simulators (e.g., AutoPOSE corpus [32]). There are very few examples of motion capture systems in naturalistic data collections in real road [27], [29], [30].…”

Section: B Multimodal Sensorsmentioning

confidence: 99%

See 1 more Smart Citation

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

Jha

Marzban

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

A smart vehicle should be able to monitor the actions and behaviors of the human driver to provide critical warnings or intervene when necessary. Recent advancements in deep learning and computer vision have shown great promise in monitoring human behavior and activities. While these algorithms work well in a controlled environment, naturalistic driving conditions add new challenges such as illumination variations, occlusions, and extreme head poses. A vast amount of in-domain data is required to train models that provide high performance in predicting driving related tasks to effectively monitor driver actions and behaviors. Toward building the required infrastructure, this paper presents the multimodal driver monitoring (MDM) dataset, which was collected with 59 subjects that were recorded performing various tasks. We use the Fi-Cap device that continuously tracks the head movement of the driver using fiducial markers, providing frame-based annotations to train head pose algorithms in naturalistic driving conditions. We ask the driver to look at predetermined gaze locations to obtain accurate correlation between the driver's facial image and visual attention. We also collect data when the driver performs common secondary activities such as navigation using a smart phone and operating the in-car infotainment system. All of the driver's activities are recorded with high definition RGB cameras and a time-of-flight depth camera. We also record the controller area network-bus (CAN-Bus), extracting important information. These high quality recordings serve as the ideal resource to train various efficient algorithms for monitoring the driver, providing further advancements in the field of in-vehicle safety systems.

show abstract

“…Survey of existing car and driving datasets indicates that the most common application for such datasets is driver monitoring. Therefore, many datasets exist for tasks like driver distraction recognition [10], driver behavior recognition [20], driver gaze detection [25,24], driver activity recognition [22] and driver intention prediction [18]. All these datasets provide color images of driver from front view for predicting driver activity or intention, with the exception of HEH and Dive&Act which provide depth images as well.…”

Section: Related Workmentioning

confidence: 99%

TICaM: A Time-of-flight In-car Cabin Monitoring Dataset

Katrolia,

Mirbach,

El-Sherif

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present TICaM, a Time-of-flight In-car Cabin Monitoring dataset for vehicle interior monitoring using a single wide-angle depth camera. Our dataset addresses the deficiencies of currently available in-car cabin datasets in terms of the ambit of labeled classes, recorded scenarios and provided annotations; all at the same time. We record an exhaustive list of actions performed while driving and provide for them multi-modal labeled images (depth, RGB and IR), with complete annotations for 2D and 3D object detection, instance and semantic segmentation as well as activity annotations for RGB frames. Additional to real recordings, we provide a synthetic dataset of in-car cabin images with same multi-modality of images and annotations, providing a unique and extremely beneficial combination of synthetic and real data for effectively training cabin monitoring systems and evaluating domain adaptation approaches. The dataset is available at https://viztatof.kl.dfki.de/.

show abstract

AutoPOSE: Large-scale Automotive Driver Head Pose and Gaze Dataset with Deep Head Orientation Baseline

Cited by 23 publications

References 11 publications

From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

TICaM: A Time-of-flight In-car Cabin Monitoring Dataset

Contact Info

Product

Resources

About