User-Defined Voice and Mid-Air Gesture Commands for Maneuver-based Interventions in Automated Vehicles

How to Interact with a Fully Autonomous Vehicle: Naturalistic Ways for Drivers to Intervene in the Vehicle System While Performing Non-Driving Related Tasks

“…Yet drivers engaged in NDRTs still need to interact with the vehicle [ 10 ] (e.g., adjust the infotainment system) or intervene to assume control (e.g., instruct the vehicle to pass a slower car ahead) [ 11 , 12 , 13 ]. Further, different NDRTs demand of drivers different physical and cognitive loads [ 9 , 14 , 15 ], which are also influenced by differences between maneuver- and nonmaneuver-based intervention scenarios [ 16 , 17 ]. Consequently, conventional touch and voice interactions may not be sufficient to accommodate these changes in physical and cognitive load thus do not adequately support drivers to perform different intervention scenarios.…”

“…With increasing levels of vehicle autonomy, maneuver control, such as steering, braking, or stopping, is still needed, though not at all times, and AVs can still hand over control to the driver [ 32 , 33 , 34 ]. In a Level 5 (SAE) vehicle, neither traditional maneuver control nor takeover ability exists [ 13 , 16 ]. However, having the ability to control the vehicle was identified as a leading factor in AV acceptance [ 15 , 20 , 28 , 35 , 36 ].…”

“…For lower-level automation (SAE Levels 2 and 3) [ 1 ], maneuver-based driving was introduced to keep the driver in the loop by controlling or cooperating with the vehicle to guide and handle system limitations rather than hand over control [ 38 , 39 ]. Similarly, in higher-level automation, maneuver-based intervention has been proposed as the way for drivers to communicate with the FAV in order to intervene in the automated system to execute the desired maneuver, such as suddenly stopping the vehicle [ 13 , 16 , 40 ]. Maneuver-based intervention improves trust and acceptance of the AV by keeping the driver in control [ 13 , 41 ], yet applying these interventions in FAVs requires new input control channels to execute the command, as steering wheels and pedals are not designed to be used for this purpose.…”

“…Maneuver-based intervention improves trust and acceptance of the AV by keeping the driver in control [ 13 , 41 ], yet applying these interventions in FAVs requires new input control channels to execute the command, as steering wheels and pedals are not designed to be used for this purpose. Previous studies [ 16 , 17 , 42 ] have discussed intervention scenarios where the driver wants to spontaneously influence FAV behavior (e.g., stop the car to pick up a friend, overtake the car ahead, or select a parking lot). These studies have focused on intervention scenarios based solely on controlling vehicle driving (maneuver-based driving), which, unfortunately, may not cover all possible interaction aspects [ 43 ].…”

“…Even though performing NDRTs was found as a major advantage of FAVs [ 36 , 48 ], previous studies have not considered how different NDRTs affect drivers’ acceptance of input interaction [ 16 , 17 ]. In manual driving, where driving itself is the primary task, typical input interactions (e.g., buttons, touch, voice, and hand gestures) are designed to increase safety rather than improve performance [ 21 , 23 , 45 , 49 ].…”