Automatic Analysis of Pedestrian’s Body Language in the Interaction with Autonomous Vehicles

A Spatio-Temporal Multilayer Perceptron for Gesture Recognition

“…traffic control officers and pedestrians [23], [24]. Further studies address other aspects of the problem, such as the lack of public datasets on traffic control gestures [5], relying on robust motion capture sensors [25], driver behavior prediction [26], and pedestrian intention prediction [27], [28], [29]. These works shed light on the importance of an accurate understanding of the human body language for automated driving.…”

Response of Vulnerable Road Users to Visual Information from Autonomous Vehicles in Shared Spaces

“…In order to obtain behavioral patterns of different individuals in the environment, we applied the algorithms described in [7] and identified poses adopted by pedestrians in the urban environment when they were exposed to the presence of a driving AV. The environment consisted of a shared space in which segregation of VRUs and vehicles was minimized.…”

“…Relying on the approach presented in [7], the specific pose of a pedestrian was identified using the OpenPose open source library developed by CMU-Panoptic labs [27], [28], [29], which designed and trained a feedback convolutional neural network that determined key points of individual poses in an RGB image and rendered the poses as seen in Figure 2. The neural network is in charge of calculating the heatmaps where the keypoints of the pose are most likely to be found, and it connects them using the Part Affinity Fields (PAFs) feature that preserves the location and orientation of people's joints.…”

“…To address this, we investigate interaction strategies by applying in field tests the algorithms for the automatic analysis of pedestrian body language presented in [7] to determine the impact of the coexistence of AVs and other road users on general road safety. To this end we focus on crossing behavior that is relatively close and directly in front of the AV, as it is relevant for safety and proves the pedestrians trust the technology.…”

Multi-Object Tracking with Data Association by a Similarity Identification Model

“…Agent's skeleton attribute has been also used to identify and track people in sequences captured from ground vehicles, with the purpose of analysing pedestrians' body language, as part of an automated tool to analyse the autonomous vehicle-pedestrian interaction. This tool is detailed described in [241], and the methods proposed in this thesis were used to develop its tracking module, as the proof of its versatility. In the context of analysing the Pedestrian's Body Language in the Interaction with Autonomous Vehicles, the tracking of the pedestrians is essential to categorise their movements for later analysis or their intention of crossing the road.…”