Toward Minimum Startle After Take-Over Request: A Preliminary Study of Physiological Data

Pakdamanian, Erfan; Namaky, Nauder; Sheng, Shili; Kim, Inki; Coan, James A.; Lu, Feng

doi:10.1145/3409251.3411715

Cited by 11 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies performed in driving simulators are beneficial in understanding causal analysis as different factors can be controlled [38]. Previous studies have coupled driving simulators with physiological sensors to detect driver states in different contextual settings such as detecting states of being awake versus sleeping in four-hour driving epochs [39], cognitive distraction when the lead vehicle abruptly breaks [40], response to automation takeover when the take over request is offered through different sources [41], and driver's performance and possible cognitive distraction when being exposed to different traffic signs [42]. Driving simulators are safe tools for conducting driving experiments in different environmental conditions, such as crash events, or evaluating drivers' emotional states [43].…”

Section: A Driving Simulatormentioning

confidence: 99%

“…Driving simulators are safe tools for conducting driving experiments in different environmental conditions, such as crash events, or evaluating drivers' emotional states [43]. Additionally, by using driving simulators, it is viable to collect modalities of data that are not feasible in a real-world setting such as brain activity signals (e.g., EEG) [41]. However, due to its controlled nature, driving simulators cannot be used to evaluate longitudinal behavioral changes.…”

Section: A Driving Simulatormentioning

confidence: 99%

See 1 more Smart Citation

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

et al. 2021

View full text Add to dashboard Cite

Effective shared autonomy requires a clear understanding of driver's behavior, which is governed by multiple psychophysiological and environmental variables. Disentangling this intricate web of interactions requires understanding the driver's state and behaviors in different real-world scenarios, longitudinally. Naturalistic Driving Studies (NDS) have shown to be an effective approach to understanding the driver's state and behavior in real-world scenarios. However, due to the lack of technological and computing capabilities, former NDS only focused on vision-based approaches, ignoring important psychophysiological factors such as cognition and emotion. The main objective of this paper is to introduce HARMONY, a human-centered multimodal naturalistic driving study, where driver's behaviors and states are monitored through (1) in-cabin and outside video streams (2) physiological signals including driver's heart rate and hand acceleration (IMU data), (3) ambient noise, light, and the vehicle's GPS location, and (4) music logs, including song features such as tempo. HARMONY is the first study that collects long-term naturalistic facial, physiological, and environmental data simultaneously. This paper summarizes HARMONY's goals, framework design, data collection and analysis, and the on-going and future research efforts. Through a presented case study, we first demonstrate the importance of longitudinal driver state sensing through using Kernel Density Estimation Methods. Second, we leverage the application of Bayesian Change Point detection methods to demonstrate how we can identify driver behaviors and responses to the environmental conditions by fusing psychophysiological information with features extracted from video streams.INDEX TERMS Naturalistic driving study, physiological sensing,driver state detection, shared-autonomy, contextual awareness, human-in-the-loop systems

show abstract

Section: A Driving Simulatormentioning

confidence: 99%

Section: A Driving Simulatormentioning

confidence: 99%

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Recent studies have provided insights into the application of wearable devices and ubiquitous computing in driving studies. These studies primarily focused on various states of the driver such as stress [17], [18], drowsiness [19], [17], distraction [20], fatigue [17], as well as different driving behaviors such as take-over readiness [21], [22], [23], driving maneuvers (e.g., turning) [24], turning and lane changes [25].…”

Section: Background Studymentioning

confidence: 99%

Driver State and Behavior Detection Through Smart Wearables

Tavakoli¹,

Kumar²,

Boukhechba³

et al. 2021

Preprint

View full text Add to dashboard Cite

Integrating driver, in-cabin, and outside environment's contextual cues into the vehicle's decision making is the centerpiece of semi-automated vehicle safety. Multiple systems have been developed for providing context to the vehicle, which often rely on video streams capturing drivers' physical and environmental states. While video streams are a rich source of information, their ability in providing context can be challenging in certain situations, such as low illuminance environments (e.g., night driving), and they are highly privacy-intrusive. In this study, we leverage passive sensing through smartwatches for classifying elements of driving context. Specifically, through using the data collected from 15 participants in a naturalistic driving study, and by using multiple machine learning algorithms such as random forest, we classify driver's activities (e.g., using phone and eating), outside events (e.g., passing intersection and changing lane), and outside road attributes (e.g., driving in a city versus a highway) with an average F1 score of 94.55, 98.27, and 97.86 % respectively, through 10-fold cross-validation. Our results show the applicability of multimodal data retrieved through smart wearable devices in providing context in real-world driving scenarios and pave the way for a better shared autonomy and privacy-aware driving data-collection, analysis, and feedback for future autonomous vehicles.

show abstract

“…Psychophysiology refers to psychological states such as emotional responses (e.g., anger, frustration, and happiness), cognitive load, and distraction, which can be measured through changes in human physiological responses (e.g., HR, skin temperature, and skin conductance) [17]. In the driving research, psychophysiological measures such as driver's HR [18]- [20], gaze patterns [10], [21], skin conductance [22], and brain signals [23] were all used for retrieving driver's stress level, and cognitive load.…”

Section: A Driver State Detectionmentioning

confidence: 99%

Multimodal Driver State Modeling through Unsupervised Learning

Tavakoli¹,

Heydarian²

2021

Preprint

View full text Add to dashboard Cite

Naturalistic driving data (NDD) can help understand drivers' reactions to each driving scenario and provide personalized context to driving behavior. However, NDD requires a high amount of manual labor to label certain driver's state and behavioral patterns. Unsupervised analysis of NDD can be used to automatically detect different patterns from the driver and vehicle data. In this paper, we propose a methodology to understand changes in driver's physiological responses within different driving patterns. Our methodology first decomposes a driving scenario by using a Bayesian Change Point detection model. We then apply the Latent Dirichlet Allocation method on both driver state and behavior data to detect patterns. We present two case studies in which vehicles were equipped to collect exterior, interior, and driver behavioral data. Four patterns of driving behaviors (i.e., harsh brake, normal brake, curved driving, and highway driving), as well as two patterns of driver's heart rate (HR) (i.e., normal vs. abnormal high HR), and gaze entropy (i.e., low versus high), were detected in these two case studies. The findings of these case studies indicated that among our participants, the drivers' HR had a higher fraction of abnormal patterns during harsh brakes, accelerating and curved driving. Additionally, free-flow driving with close to zero accelerations on the highway was accompanied by more fraction of normal HR as well as a lower gaze entropy pattern. With the proposed methodology we can better understand variations in driver's psychophysiological states within different driving scenarios. The findings of this work, has the potential to guide future autonomous vehicles to take actions that are fit to each specific driver.

show abstract

Toward Minimum Startle After Take-Over Request: A Preliminary Study of Physiological Data

Cited by 11 publications

References 13 publications

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

Driver State and Behavior Detection Through Smart Wearables

Multimodal Driver State Modeling through Unsupervised Learning

Contact Info

Product

Resources

About