Driver behavior recognition based on deep convolutional neural networks

Yan, Shiyang; Teng, Yuxuan; Smith, Jeremy S.; Zhang, Bailing

doi:10.1109/fskd.2016.7603248

Cited by 57 publications

(41 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two are the most utilized approaches: detecting inattention by monitoring the eyes and head orientation of the driver [e.g. 2,3,7,19], and activity recognition through hand tracking [31,34,35]. Across all the studies that deal with eye and head orientation, tracking is performed using a camera, and then, based on the collected images, a machine learning algorithm asserts the level of driver's attentiveness.…”

Section: Activity Recognition In a Carmentioning

confidence: 99%

“…In more detail, such systems were used to detect if drivers are looking at the road [1], or how tired they are [9,11]. In relation to hand tracking, the authors of [31,34,35] managed to assert drivers' activities using depth cameras. In most of these studies, the camera is mounted on the side of the vehicle and the activities performed by the driver are classified using machine learning.…”

Section: Activity Recognition In a Carmentioning

confidence: 99%

“…21], and through hand tracking using depth cameras [e.g. 18,31,34,35]. In relation to warning mechanisms, various modalities have been utilized such as visual [e.g.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dara

Raptis

Iversen

Mølbak

et al. 2018

Proceedings of the 10th Nordic Conference on Human-Computer Interaction

View full text Add to dashboard Cite

This paper presents DARA, the Driving Awareness and Reflection Assistant that makes drivers aware of potentially dangerous practices on how they hold the steering wheel, and helps them reflect. DARA utilizes a hand recognition component and a feedback one. The first recognizes how drivers hold the steering wheel and classifies their actions through a Leap Motion controller and machine learning. The second is comprised by a mobile application that provides drivers with feedback during and after their drive. DARA was evaluated with three studies for its accuracy, relevance and utility. Our findings show that DARA was successful both in making holding patterns present-at-hand for the drivers and in assisting them to reflect. We conclude our paper with a discussion on the broader implication of our findings on in-car hand recognition and feedback systems.

show abstract

Section: Activity Recognition In a Carmentioning

confidence: 99%

Section: Activity Recognition In a Carmentioning

confidence: 99%

See 1 more Smart Citation

Dara

Raptis

Iversen

Mølbak

et al. 2018

Proceedings of the 10th Nordic Conference on Human-Computer Interaction

View full text Add to dashboard Cite

show abstract

“…Recent related works involving driver behaviour include a paper by Yan [23], using the Southeast University Driving-posture Dataset (SEU dataset) by [25] designed a CNN model to identify 6 driver actions: calling, eating, braking, wheel use, phone use, and smoking. They used 2 inputs to the CNN, a primary input and a secondary input; the primary input was a bounding box around the whole driver in the image, and the secondary input was a set of skin regions.…”

Section: Introductionmentioning

confidence: 99%

“…An action can be determined from contextual cues, such as a person's pose or the presence of an object. For example, Gkioxari's [7] action recognition method exploits this by marking bounding boxes around the subject and bounding boxes around relevant contextual cues and then using these as inputs to a CNN, in a manner similar to Yan's driver monitoring method [23]. Because passenger state monitoring and action detection is still in its infancy and focused on the driver, there is limited data available depicting passenger state, and much of this data relevant for driver behaviour, and action classification pertinent to vehicle safety.…”

Section: Introductionmentioning

confidence: 99%

Deep Passenger State Monitoring Using Viewpoint Warping

Bhalerao

Griffiths

et al. 2017

Image Analysis and Processing - ICIAP 2017

View full text Add to dashboard Cite

Abstract. The advent of autonomous and semi-autonomous vehicles has meant passengers now play a more significant role in the safety and comfort of vehicle journeys. In this paper, we propose a deep learning method to monitor and classify passenger state with camera data. The training of a convolutional neural network is supplemented by data captured from vehicle occupants in different seats and from different viewpoints. Existing driver data or data from one vehicle is augmented by viewpoint warping using planar homography, which does not require knowledge of the source camera parameters, and overcomes the need to re-train the model with large amounts of additional data. To analyse the performance of our approach, data is collected on occupants in two different vehicles, from different viewpoints inside the vehicle. We show that the inclusion of the additional training data and augmentation by homography increases the average passenger state classification rate by 11.1%. We conclude by proposing how occupant state may be used holistically for activity recognition and intention prediction for intelligent vehicle features.

show abstract

Computer vision‐based recognition of driver distraction: A review

Moslemi

Soryani

Azmi

2021

Concurrency and Computation

View full text Add to dashboard Cite

Vehicle crash rates caused by distracted driving have been rising in recent years. Hence, safety while driving on roads is today a crucial concern across the world. Some of the reasons due to which drivers may lose attention include the use of mobile phones, speaking with passengers, and reaching behind to grab something while driving. There are various types of distractions, out of which we focus on manual ones based on the posture of the driver. This work presents a review on open problems and challenges concerning the vision-based distraction recognition of drivers. Types of experimental environments are described; and image modalities, public and custom datasets and body parts inspected during distraction recognition are explored. Besides, a comparative review of different works on vision-based recognition of driver's manual distractions and limitations for each dataset is presented. Main approaches of vision-based manual distractive driving recognition can be categorized into conventional and deep learning methods. These approaches are compared and classified based on whether using temporal information or not. Finally, we give some suggestions for improvement, and look forward to future development directions of the vision-based recognition of driver's manual distractions. K E Y W O R D Sdistraction recognition, driver dataset, driver gesture, driver monitoring, driving simulator INTRODUCTIONDistractions decrease driver's awareness and decision-making; lead to increased workload, delayed responses to driving events, speed and lane maintenance disruptions. 1 Distracted drivers would lose their vehicle control, resulting in crashing into vehicles or stationary objects. 2 In other words, such a driver loses vigilance regarding the driving situation, leading to a higher collision risk 3 ; then the safety of the driver and other passengers are affected. The National Highway Traffic Safety Administration (NHTSA) 4 has reported three activities, including dialing, browsing, and texting, which make drivers have their eyes off the road for a longer time, lead to threefold crash risk. Performing high cognitive load tasks while driving impacts the driver's visual behavior and driving performance. Drivers under high cognitive loads spend less time checking mirrors, traffic lights instruments, and areas around intersections. 5 This results in unintended lane drifts, overlooking other traffic participants and, in the worst scenario, crashes. 6 Rapid improvements in computers and wireless technologies have made complex and various devices that people can use while driving.Notably, in the field of entertainment, information and driver assistance, studies have proposed numerous functions. However, using such functions during driving could distract the driver from focusing on driving itself. 6 The widespread usage of cell phones, navigation, and infotainment systems has escalated the problem of distraction. Therefore, the cognitive load caused by secondary tasks has increased. Multitasking is today an essential element of ind...

show abstract

Driver behavior recognition based on deep convolutional neural networks

Cited by 57 publications

References 27 publications

Dara

Dara

Deep Passenger State Monitoring Using Viewpoint Warping

Computer vision‐based recognition of driver distraction: A review

Contact Info

Product

Resources

About