Understanding pedestrian behaviour with pose estimation and recurrent networks

Marginean, Anca; Brehar, Raluca; Negru, Mihai

doi:10.1109/iseee48094.2019.9136126

Cited by 11 publications

(15 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Method Accuracy Alexnet + Context [15] 63.0% Alexnet + SVM [50] 74.4% Alphapose + LSTM [56] 78.0% Res-EnDec [53] 81.0% ST-DenseNet [52] 84.76% auto-encoder + Prediction [54] 86.7% Openpose + Keypoints [55] 88.0% Alexnet + SVM + Context [50] 89.4% CPN + GCN [58] 91 Overall, although SPI-Net is not that complex in its architecture, Table 3 shows that it outperforms by more than 2.5% the current state-of-the-art approach [58] based on CPN [14] for pedestrian discrete intention prediction task on the JAAD data. The confusion matrices in Table 4 also shows that SPI-Net accuracy is similar on both action classes, which demonstrates its ability to adapt to intra-class variation for skeleton-based dynamics.…”

Section: Results On Jaad Data Setmentioning

confidence: 99%

“…Fang et al [55] combined CNN-based pedestrian detection, tracking and pose estimation to predict the crossing action from monocular images. Marginean et al [56] and Ghori et al [57] explore the pedestrian intention prediction task with pose estimation algorithms combined with recurrent networks. However, in [57], sequences in the wild are used, which makes it difficult to evaluate their approach on the JAAD data set.…”

Section: Pedestrian Intention Predictionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

et al. 2020

View full text Add to dashboard Cite

Understanding the behaviors and intentions of humans is still one of the main challenges for vehicle autonomy. More specifically, inferring the intentions and actions of vulnerable actors, namely pedestrians, in complex situations such as urban traffic scenes remains a difficult task and a blocking point towards more automated vehicles. Answering the question “Is the pedestrian going to cross?” is a good starting point in order to advance in the quest to the fifth level of autonomous driving. In this paper, we address the problem of real-time discrete intention prediction of pedestrians in urban traffic environments by linking the dynamics of a pedestrian’s skeleton to an intention. Hence, we propose SPI-Net (Skeleton-based Pedestrian Intention network): a representation-focused multi-branch network combining features from 2D pedestrian body poses for the prediction of pedestrians’ discrete intentions. Experimental results show that SPI-Net achieved 94.4% accuracy in pedestrian crossing prediction on the JAAD data set while being efficient for real-time scenarios since SPI-Net can reach around one inference every 0.25 ms on one GPU (i.e., RTX 2080ti), or every 0.67 ms on one CPU (i.e., Intel Core i7 8700K).

show abstract

Section: Results On Jaad Data Setmentioning

confidence: 99%

Section: Pedestrian Intention Predictionmentioning

confidence: 99%

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

et al. 2020

View full text Add to dashboard Cite

show abstract

“…However, the predictable change event types of this scheme are limited, and it cannot be widely used in various traffic scenarios. Anca Marginean et al [29] proposed a set of posebased and recursive framework-based algorithms to deal with imbalances in pedestrian estimation. When our scene is set at intersections and sidewalks, Keller and Gavrila [30] used the Gaussian dynamics model and probabilistic hierarchical trajectories based on dense optical flow to obtain pedestrian characteristics.…”

Section: Key Point Predictionmentioning

confidence: 99%

At the Traffic Intersection, Stopping, or Walking? Pedestrian Path Prediction Based on KPOF-GPDM for Driving Assistance

Long¹,

Zhang

Zhao³

et al. 2021

Journal of Advanced Transportation

View full text Add to dashboard Cite

Pedestrian detection has always been a research hotspot in the Advanced Driving Assistance System (ADAS) with great progress in recent years. However, for the ADAS, we not only need to detect the behavior of pedestrians in front of the vehicle but also predict future action and the motion trajectory. Therefore, in this paper, we propose a human key point combined optical flow network (KPOF-Net) in the vehicle ADAS for the occlusion situation in the actual scene. When the vehicle encounters a blocked pedestrian at a traffic intersection, we used self-flow to estimate the global optical flow in the image sequence and then proposed a White Edge Cutting (WEC) algorithm to remove obstructions and simply modified the generative adversarial network to initialize pedestrians behind the obstructions. Next, we extracted pedestrian optical flow information and human joint point information in parallel, among which we trained four human key point models suitable for traffic intersections. At last, KPOF-GPDM fusion was proposed to predict the future status and walking trajectories of pedestrians, which combined optical flow information with human key point information. In the experiment, we did not merely compare our method with other four representative approaches in the same scene sequences. We also verified the accuracy of the pedestrian motion state and motion trajectory prediction of the system after fusion of human joint points and optical flow information. Taking into account the real-time performance of the system, in the low-speed and barrier-free environment, the comparative analysis only uses optical flow information, human joint point information, and KPOF-Net three prediction models. The results show that (1) in the same traffic environment, our proposed KPOF-Net can predict the change of pedestrian motion state about 5 frames (about 0.26 s) ahead of other excellent systems; (2) at the same time, our system predicts the trajectory of the pedestrian more accurately than the other four systems, which can achieve more stable minimum error ±0.04 m; (3) in a low-speed, barrier-free experimental environment, our proposed trajectory prediction model that integrates human joint points and optical flow information has higher prediction accuracy and smaller fluctuations than a single-information prediction model, and it can be well applied to automobiles’ ADAS.

show abstract

“…Consumer behavior digital could provide brands valuable information, this information could provide consumers with better service and experience. So convenient and accurate digital information collection methods had been widely studied by people (Frontoni et al, 2013, Marginean et al, 2019.…”

Section: Introductionmentioning

confidence: 99%

Application of Openpose algorithm to detect consumer behavior in store

Li¹,

Zeng²,

Hou³

et al. 2020

Proceedings of the 8th International Conference on Advanced Materials and Systems

View full text Add to dashboard Cite

Due to the importance of offline consumer behavior, more and more people had begun to study consumer behavior in store. In offline consumer behavior research, the application of video analysis technology was the most direct and convenient. Recognizing human posture was a key technology in video analysis. The OpenPose algorithm was one of the advantageous technologies that could accurately recognize multi-person poses in different environments in real time, so we used it innovatively to study consumer behavior in store. We hope to develop the potential of this application in the research of consumer behavior in store in the footwear retail industry by the technical advantages of the OpenPose algorithm. In our study, we first used an OpenPose algorithm to estimate multi-person pose and detection behavior, and then processed and recognized the videos collected in the store. We collected a week's surveillance video of a Red Dragonfly offline store from July 10 to July 16, 2020 in China. The specific process was to calibrate the area in the selected camera screen, then the algorithm performs identification and detection, and finally output in-store consumption Behavioral data. Our research results not only verified the feasibility of this application in offline retailing stores, but the data results also indicated that consumers tend to enter the store from the right, staying concentrated in the middle and back of the store. These results may be affected by the store space, product display, and staff guidance and reception.

show abstract

Understanding pedestrian behaviour with pose estimation and recurrent networks

Cited by 11 publications

References 6 publications

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

At the Traffic Intersection, Stopping, or Walking? Pedestrian Path Prediction Based on KPOF-GPDM for Driving Assistance

Application of Openpose algorithm to detect consumer behavior in store

Contact Info

Product

Resources

About