User Monitoring in Autonomous Driving System Using Gamified Task: A Case for VR/AR In-Car Gaming

Abstract

Background: As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver’s obligation will be changed to a supervisory role. A key point to consider is the driver’s engagement in the secondary task to maintain the driver/user in the control loop. This paper aims to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. Methods: We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. Results: There was a significant difference in the tasks (F(2,33) = 4.34, p = 0.0213), identifying the game-task as significant with respect to reaction time and ideal for the present investigation. Game scoring followed three profiles/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify/infer drivers’ engagement with the game task. Conclusion: The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments with AR-Games focusing on the real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.

Feasibility of AR-VR use in autonomous cars for user engagements and its effects on posture and vigilance during transit

“…From this, we concluded that the inclusion of additional tasks did not negatively affect hazard recognition. This is in agreement with the same findings we identified in a related study reported in (Muguro et al, 2021). Research on take-over time, which hazard recognition points to, has indicated that cognitive-and visual-loaded secondary tasks do not have major impacts on take-over performance (Happee et al, 2017;Zeeb et al, 2016).…”

“…This way, vigilance can be supported for extended periods. More details on this have been discussed in (Muguro et al, 2021). In this regard, the results support the idea that ADS-human handover would be enhanced through the identified driver engagement levels, where the system only performs handover when the user is demonstrably vigilant.…”

“…We designed a custom car simulator scene with Unity 3D game engine for the proposed driver analysis. The VR content was designed to be experienced by passengers in a realworld moving car as opposed to previous design that targeted parked cars or reduced car dynamics (straight lines) (Hock et al, 2017;Muguro et al, 2021). The car in use was a Subaru Stella Matic, with a displacement of 600cc, with no special relaxation features in place.…”