Lidar-based gait analysis in people tracking and 4D visualization

Abstract

In this paper we introduce a new approach on gait analysis based on data streams of a Rotating Multi Beam (RMB) Lidar sensor. The gait descriptors for training and recognition are observed and extracted in realistic outdoor surveillance scenarios, where multiple pedestrians walk concurrently in the field of interest, while occlusions or background noise may affects the observation. The proposed algorithms are embedded into an integrated 4D vision and visualization system. Gait features are exploited in two different components of the workflow. First, in the tracking step the collected characteristic gait parameters support as biometric descriptors the re-identification of people, who temporarily leave the field of interest, and re-appear later. Second, in the visualization module, we display moving avatar models which follow in real time the trajectories of the observed pedestrians with synchronized leg movements. The proposed approach is experimentally demonstrated in eight multi-target scenes.

Feature selection for Lidar-based gait recognition

“…We have tested the above listed five methods in six outdoor sequences captured in a courtyard by a Velodyne HDL 64-E RMB Lidar sensor (sequence names in Table 1 refer to the date of capturing and the number of people in the scene) [12,14]. In all the sequences, the test subjects circulate in the scene, then they leave the FoV for a while, and re-appear later in a different order.…”

“…The first four techniques are Lidar-focused applications of state-of-the-art approaches, proposed earlier for standard optical and Kinect data, while the fifth method is an improved modification of our model from [14].…”

“…2. Extracted features of the (a) Silhouette print [6] and (b) GEI [14,15] techniques on Lidar data tions of the silhouettes. Instead, we exploit an assumption that the people walk forwards in the scene, always turning towards the tangent direction of the trajectory.…”

“…As a modification of the original GEI approach, we introduced in [14] the Lidar-based GEI technique, which trained a multilayer perceptron (MLP) for each detected person using the extracted GEI prints. As an improvement, we propose here using an ensemble of a single convolutional neural network (CNN) and a MLP, both ones having N outputs, where N is equal to the number of people in the training scenario, and each output value represents a person.…”

“…Although pedestrian detection and tracking tasks have already been conducted on RMB Lidar measurements [12,13], to our best knowledge our research [14] has been the first at- [14] at a proof-of-concept level, we focus here on comparing features used in earlier techniques, which have originally been proposed for gait analysis in optical or Kinect data. By each selected method, we had to explore first how their expected input feature maps can be derived from RMB Lidar streams.…”