DriverMVT: In-Cabin Dataset for Driver Monitoring including Video and Vehicle Telemetry Information

Othman, Walaa; Kashevnik, Alexey; Ali, Ammar; Shilov, Nikolay

doi:10.3390/data7050062

Cited by 11 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pé rez-Jimé nez et al [23] proposes a system based on analysis of visual images and combines the results of different types of classifiers to achieve robust and real-time detection. Othman et al [24] have presented a driver monitoring system using in-cabin monitoring and deep learning. They have gathered a video and telemetry dataset for driver monitoring.…”

Section: Vision Methods For In Cabin Occupant Monitoringmentioning

confidence: 99%

See 1 more Smart Citation

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

KHRAIEF OULED AZAIZ,

DACLEU NDENGUE

2023

Fifteenth International Conference on Machine Vision (ICMV 2022)

View full text Add to dashboard Cite

Vehicle in-cabin occupant monitoring system is becoming a crucial feature of the automobile industry and challenging research topic to enhance both vehicle safety, security, and comfort of conventional and future intelligent vehicles. Precise information about the number, position, and characteristics of occupants as well as objects located inside the vehicle must be available. Current industrial systems for seat occupancy detection are based on multiple weight sensors, capacitive sensors, electric field, or ultrasonic sensors. They cannot necessarily make the right distinction in borderline cases. A simple pressure sensor cannot tell whether the weight on the seat comes from a person or an inanimate object. Recently, the Artificial Intelligence (AI) based advanced systems have attracted attention for various fields such as automobile industry. Especially, with the advancement of deep learning that has shown very high classification accuracies compared to hand-crafted features on various computer vision tasks. For the above reasons, we propose a new automatic AI occupant monitoring system based on two cameras installed inside the vehicle. Our goal is to develop an automatic detection and recognition system with high accuracy performance, low computational cost and small weight model. Our system fuses our modified deep convolutional network Yolo model and deep reinforcement learning to detect and classify passengers and objects inside the vehicle. It can predict the gender, the age and the emotion of occupants based on our proposed muti task convolutional neural networks. In our end-to-end system, this approach is more efficient time and memory wise by solving all the tasks in the same process and storing a single CNN instead of storing a CNN for each task. Principal applications of our system are intelligent airbag management, seat belt reminder, life presence and in shared cabin preferences. We perform comparative evaluation based on the public datasets SVIRO, TiCaM, Aff-Wild and Adience dataset to demonstrate the superior performance of our proposed system.

show abstract

Section: Vision Methods For In Cabin Occupant Monitoringmentioning

confidence: 99%

“…This dataset consists of synthetic and real annotated images for monitoring front seats. The Adience benchmark [24] is used to train and evaluate the Muti-task CNN model. this dataset is constructed for age and gender recognition.…”

Section: Datasetsmentioning

confidence: 99%

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

KHRAIEF OULED AZAIZ,

DACLEU NDENGUE

2023

Fifteenth International Conference on Machine Vision (ICMV 2022)

View full text Add to dashboard Cite

show abstract

“…Pappalardo et al have applied the decision tree method to analyze the lane support system in [38]. Othman et al have presented a driver monitoring system using in-cabin monitoring and deep learning [39]. They have gathered a video and telemetry dataset for driver monitoring.…”

Section: Related Workmentioning

confidence: 99%

In-Cabin Monitoring System for Autonomous Vehicles

Mishra

Lee

Kim

et al. 2022

Sensors

View full text Add to dashboard Cite

In this paper, we have demonstrated a robust in-cabin monitoring system (IMS) for safety, security, surveillance, and monitoring, including privacy concerns for personal and shared autonomous vehicles (AVs). It consists of a set of monitoring cameras and an onboard device (OBD) equipped with artificial intelligence (AI). Hereafter, this combination of a camera and an OBD is referred to as the AI camera. We have investigated the issues for mobility services in higher levels of autonomous driving, what needs to be monitored, how to monitor, etc. Our proposed IMS is an on-device AI system that indigenously has improved the privacy of the users. Furthermore, we have enlisted the essential actions to be considered in an IMS and developed an appropriate database (DB). Our DB consists of multifaced scenarios important for monitoring the in-cabin of the higher-level AVs. Moreover, we have compared popular AI models applied for object and occupant recognition. In addition, our DB is available on request to support the research on the development of seamless monitoring of the in-cabin higher levels of autonomous driving for the assurance of safety and security.

show abstract

“…• Driver behavior tracking that, for example, aimed at detection of the driver's drowsiness [77] and distraction [78], unfastened belt [79], etc. ; • Road situation tracking that, for example, aims at detection of road accidents [80], technical works [81], specific weather conditions [82], etc.…”

Section: Related Workmentioning

confidence: 99%

Security and Privacy Analysis of Smartphone-Based Driver Monitoring Systems from the Developer’s Point of View

Levshun

Chechulin

Kotenko

2022

Sensors

View full text Add to dashboard Cite

Nowadays, the whole driver monitoring system can be placed inside the vehicle driver’s smartphone, which introduces new security and privacy risks to the system. Because of the nature of the modern transportation systems, the consequences of the security issues in such systems can be crucial, leading to threat to human life and health. Moreover, despite the large number of security and privacy issues discovered in smartphone applications on a daily basis, there is no general approach for their automated analysis that can work in conditions that lack data and take into account specifics of the application area. Thus, this paper describes an original approach for a security and privacy analysis of driver monitoring systems based on smartphone sensors. This analysis uses white-box testing principles and aims to help developers evaluate and improve their products. The novelty of the proposed approach lies in combining various security and privacy analysis algorithms into a single automated approach for a specific area of application. Moreover, the suggested approach is modular and extensible, takes into account specific features of smartphone-based driver monitoring systems and works in conditions of lack or inaccessibility of data. The practical significance of the approach lies in the suggestions that are provided based on the conducted analysis. Those suggestions contain detected security and privacy issues and ways of their mitigation, together with limitations of the analysis due to the absence of data. It is assumed that such an approach would help developers take into account important aspects of security and privacy, thus reducing related issues in the developed products. An experimental evaluation of the approach is conducted on a car driver monitoring use case. In addition, the advantages and disadvantages of the proposed approach as well as future work directions are indicated.

show abstract

DriverMVT: In-Cabin Dataset for Driver Monitoring including Video and Vehicle Telemetry Information

Cited by 11 publications

References 19 publications

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

In-Cabin Monitoring System for Autonomous Vehicles

Security and Privacy Analysis of Smartphone-Based Driver Monitoring Systems from the Developer’s Point of View

Contact Info

Product

Resources

About