I See Your Gesture: A VR-Based Study of Bidirectional Communication between Pedestrians and Automated Vehicles

Abstract

Automated vehicles (AVs) are able to detect pedestrians reliably but still have difficulty in predicting pedestrians’ intentions from their implicit body language. This study examined the effects of using explicit hand gestures and receptive external human-machine interfaces (eHMIs) in the interaction between pedestrians and AVs. Twenty-six participants interacted with AVs in a virtual environment while wearing a head-mounted display. The participants’ movements in the virtual environment were visualized using a motion-tracking suit. The first independent variable was the participants’ opportunity to use a hand gesture to increase the probability that the AV would stop for them. The second independent variable was the AV’s response “I SEE YOU,” displayed on an eHMI when the vehicle yielded. Accordingly, one-way communication (gesture or eHMI) and two-way communication (gesture and eHMI combined) were investigated. The results showed that the participants decided to use hand gestures in 70% of the trials. Furthermore, the eHMI improved the predictability of the AV’s behavior compared to no eHMI, as inferred from self-reports and hand-use behavior. A postexperiment questionnaire indicated that two-way communication was the most preferred condition and that the eHMI alone was more preferred than the gesture alone. The results further indicate limitations of hand gestures regarding false-positive detection and confusion if the AV decides not to yield. It is concluded that bidirectional human-robot communication has considerable potential.

Designing Wearable Augmented Reality Concepts to Support Scalability in Autonomous Vehicle-Pedestrian Interaction

“…Concerning interactions with AVs in safety-critical settings, the participants unanimously agreed on the need to make their crossing intentions known to AVs. This finding is consistent with a prior study on bidirectional communication between pedestrians and AVs (Colley et al, 2021;Epke et al, 2021), which showed that a combination of hand gestures and receptive eHMIs was the most desired method of communication. However, while hand gestures have been previously observed to have limitations in terms of false-positive (Epke et al, 2021) or false-negative detection (Gruenefeld et al, 2019), a digital approach was viewed as safer and more trustworthy in our study.…”

“…This finding is consistent with a prior study on bidirectional communication between pedestrians and AVs (Colley et al, 2021;Epke et al, 2021), which showed that a combination of hand gestures and receptive eHMIs was the most desired method of communication. However, while hand gestures have been previously observed to have limitations in terms of false-positive (Epke et al, 2021) or false-negative detection (Gruenefeld et al, 2019), a digital approach was viewed as safer and more trustworthy in our study. Additionally, using wearable AR for bidirectional communication not only ensures that AVs accurately interpret pedestrian intentions but might also eliminate potential confusion about AV non-yielding behaviors (Epke et al, 2021).…”

“…However, while hand gestures have been previously observed to have limitations in terms of false-positive (Epke et al, 2021) or false-negative detection (Gruenefeld et al, 2019), a digital approach was viewed as safer and more trustworthy in our study. Additionally, using wearable AR for bidirectional communication not only ensures that AVs accurately interpret pedestrian intentions but might also eliminate potential confusion about AV non-yielding behaviors (Epke et al, 2021). For example, AR may be utilized to increase system transparency by explaining long wait times or a refusal to yield.…”

Augmented reality for supporting the interaction between pedestrians and automated vehicles: an experimental outdoor study

“…This method for measuring crossing intentions was previously introduced by De Clercq et al (2019) and was considered favourable as it allows participants to respond almost immediately to changing conditions. In other research, we had participants actually cross ( Tabone et al, 2023b ) or communicate their crossing intention or ‘critical gap’ through hand gestures ( Rodríguez Palmeiro et al, 2018 ) or by taking a single step ( Epke et al, 2021 ). A drawback of these methods is the variability among participants in executing these actions, making it relatively challenging to extract their intentions or perceptions.…”

"It Must Be Gesturing Towards Me": Gesture-Based Interaction between Autonomous Vehicles and Pedestrians

“…2), have received comparatively less attention as cues to be referred to in existing designs and research on eHMI, especially compared with lights and texts. However, the previous studies conducted on gesture-related interactions between pedestrians and autonomous vehicles have mostly focused on the AV's detection of pedestrian intents via identifying and understanding their gestures [25]. The main objective of this study is to fill in the existing research gap about gesture-based eHMI to facilitate interactions between AVs and pedestrians.…”