Multimodal Cue Combinations: A Possible Approach to Designing In-Vehicle Takeover Requests for Semi-autonomous Driving

Abstract

The rapid growth of autonomous vehicles is expected to improve roadway safety. However, certain levels of vehicle automation will still require drivers to ‘takeover’ during abnormal situations, which may lead to breakdowns in driver-vehicle interactions. To date, there is no agreement on how to best support drivers in accomplishing a takeover task. Therefore, the goal of this study was to investigate the effectiveness of multimodal alerts as a feasible approach. In particular, we examined the effects of uni-, bi-, and trimodal combinations of visual, auditory, and tactile cues on response times to takeover alerts. Sixteen participants were asked to detect 7 multimodal signals (i.e., visual, auditory, tactile, visual-auditory, visual-tactile, auditory-tactile, and visual-auditory-tactile) while driving under two conditions: with SAE Level 3 automation only or with SAE Level 3 automation in addition to performing a road sign detection task. Performance on the signal and road sign detection tasks, pupil size, and perceived workload were measured. Findings indicate that trimodal combinations result in the shortest response time. Also, response times were longer and perceived workload was higher when participants were engaged in a secondary task. Findings may contribute to the development of theory regarding the design of takeover request alert systems within (semi) autonomous vehicles.

Open-Loop Naturalistic Driving: An Assessment of Human Behavior, Performance, and Physiology to Support the Development of Shared Control Strategies

“…A two-way repeated-measures ANOVA was conducted on the 4 obstacle types (tire, construction zone, rain and wind, deer) x 6 NASA-TLX un-weighted subscales (mental demand, physical demand, temporal demand, performance, effort, and frustration) (similar to Huang et al, 2019). There was a significant main effect of obstacle type (F(1, 15) = 28.967, p < .001, partial η 2 = 0.659) and subscale (F(1, 15) = 10.325, p = .006, partial η 2 = 0.408) on workload (see Figure 4).…”

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

“…Clark and Feng [56] report lower TOTs for the young group for a TOT budget of 4.5s, and lower TOTs for the old group for a 7.5s TOT budget. TOR modality: Petermeijer et al [57] and Huang et al [58] compare different modalities for issuing the TOR. Auditory and tactile TORs are considered in [57] while auditory, tactile and visual TORs and their combinations are considered in [58].…”

“…TOR modality: Petermeijer et al [57] and Huang et al [58] compare different modalities for issuing the TOR. Auditory and tactile TORs are considered in [57] while auditory, tactile and visual TORs and their combinations are considered in [58]. Both studies report the lowest TOTs for multimodal TORs.…”

Driver Logo Sign Detection and Hazard Responses during Partially Automated Driving

“…Although automated vehicles might be capable of sensing, collecting, integrating and processing a large volume of roadway condition information, as well as negotiating some operating situations, many domain experts still have doubts about safety. Research has indicated that such technologies can introduce additional safety risks, as the driver is disconnected from driving tasks and there may be vehicle conditions and road environments that are unmanageable by automation (Kockelman et al, 2016;Koopman & Wagner, 2017;ITF, 2018;Huang et al, 2019). Therefore, a driver may need to "takeover" vehicle control in certain hazard situations, including lost GPS signals, unclear and/or missing lane markings, construction zone entry points or road closures, and high traffic density (Körber, Prasch, & Bengler, 2018;Molnar et al, 2017).…”