Onboard monocular pedestrian detection by combining spatio-temporal hog with structure from motion algorithm

Abstract

In this paper, we brought out a novel pedestrian detection framework for the advanced driver assistance system of mobile platform under the normal urban street environment. Different from the conventional systems that focus on the pedestrian detection at near distance by interfusing multiple sensors (such as radar, laser and infrared camera), our system has achieved the pedestrian detection at all (near, middle and long) distance on a normally driven vehicle (1-40 km/h) with monocular camera under the street scenes. Since pedestrians typically exhibit not only their humanlike shape but also the unique human movements generated by their legs and arms, we use the spatio-temporal histogram of oriented gradient (STHOG) to describe the pedestrian appearance and motion features. The shape and movement of a pedestrian will be described by a unique feature produced by concatenating the spatial and temporal his- (1) ground constraint with monocular camera to reduce the computational cost and false alarms; (2) preprocessing by stabilizing the successive images captured from mobile camera with the SfM algorithm; (3) long-distance (maximum 100 m) pedestrian detection at various velocities (1-40 km/h). Through the extensive experiments under different city scenes, the effectiveness of our algorithm has been proved.

Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding

“…Compared to traditional methods that use spatio-temporal features to describe human motion (such as HOGHOF [32], STHOG [33,34], STGGP [35] and 3DHOG [36]), recently developed GCNs can recognize human actions more accurately by using the key points extracted from their bodies as the input features. As illustrated in Figure 11, after obtaining the key skeleton points from a real UAV pilot, all of these sequential key points will be transmitted to the spatial-temporal GCN (ST-GCN [19]) module for the action recognition task.…”

Bio-inspired heterogeneous architecture for real-time pedestrian detection applications

“…Another approach is to augment HOG descriptors with temporal informations obtained from video sequences. In [15], for example, the authors aggregate several descriptors obtained by different techniques to extract temporal information from images. The 3DHOG descriptor proposed in [21] for characterizing motion features with a co-occurence spatio-temporal vector also belongs to this category.…”

Online control programming algorithm for human–robot interaction system with a novel real-time human gesture recognition method

“…Compared with other human communication methods like facial expression, eye tracking, or head movement, human gestures are more easily understood, 18,19 and more complex information can be expressed with different gestures (as shown in Figure 2). Vision sensors are seldom affected by such electromagnetic interference, and vision-based methods have the advantages of convenient interaction, rich expression, and interactive nature.…”