Data‐Driven Crowd Motion Control With Multi‐Touch Gestures

Shen, Yijun; Henry, Joseph; Wang, He; Ho, Edmond S. L.; Komura, Taku; Shum, Hubert P. H.

doi:10.1111/cgf.13333

Cited by 11 publications

(8 citation statements)

References 46 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical modelling and data-driven methods have been the two foundations in simulation [40,35]. Early research is dominated by empirical modelling or rulebased methods, where crowd motions are abstracted into mathematical equations and deterministic systems, such as flows [40], particle systems [20], and velocity and geometric optimization [8,52]. Meanwhile, data-driven methods using statistical machine learning have also been employed, e.g., using first-person vision to guide steering behaviors [35] or using trajectories to extract features to describe motions [23,68].…”

Section: Pedestrian and Crowd Simulationmentioning

confidence: 99%

Human Trajectory Prediction via Neural Social Physics

Yue¹,

Manocha²,

Wang³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Trajectory prediction has been widely pursued in many fields, and many model-based and model-free methods have been explored. The former include rule-based, geometric or optimization-based models, and the latter are mainly comprised of deep learning approaches. In this paper, we propose a new method combining both methodologies based on a new Neural Differential Equation model. Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters. The explicit physics model serves as a strong inductive bias in modeling pedestrian behaviors, while the rest of the network provides a strong data-fitting capability in terms of system parameter estimation and dynamics stochasticity modeling. We compare NSP with 15 recent deep learning methods on 6 datasets and improve the state-of-the-art performance by 5.56%-70%. Besides, we show that NSP has better generalizability in predicting plausible trajectories in drastically different scenarios where the density is 2-5 times as high as the testing data. Finally, we show that the physics model in NSP can provide plausible explanations for pedestrian behaviors, as opposed to black-box deep learning. Code is available: https://github.com/realcrane/Human-Trajectory-Prediction-via-Neural-Social-Physics.

show abstract

Section: Pedestrian and Crowd Simulationmentioning

confidence: 99%

Human Trajectory Prediction via Neural Social Physics

Yue¹,

Manocha²,

Wang³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Learning a universal model from all cars with fine details is therefore highly ineffective. Motivated by the success of lazy learning in mesh processing [3,20,45], we propose to adapt lazy learning to reconstruct the details.…”

Section: Lazy Learning For Fine Detailsmentioning

confidence: 99%

3D car shape reconstruction from a contour sketch using GAN and lazy learning

et al. 2021

Self Cite

View full text Add to dashboard Cite

Abstract3D car models are heavily used in computer games, visual effects, and even automotive designs. As a result, producing such models with minimal labour costs is increasingly more important. To tackle the challenge, we propose a novel system to reconstruct a 3D car using a single sketch image. The system learns from a synthetic database of 3D car models and their corresponding 2D contour sketches and segmentation masks, allowing effective training with minimal data collection cost. The core of the system is a machine learning pipeline that combines the use of a generative adversarial network (GAN) and lazy learning. GAN, being a deep learning method, is capable of modelling complicated data distributions, enabling the effective modelling of a large variety of cars. Its major weakness is that as a global method, modelling the fine details in the local region is challenging. Lazy learning works well to preserve local features by generating a local subspace with relevant data samples. We demonstrate that the combined use of GAN and lazy learning produces is able to produce high-quality results, in which different types of cars with complicated local features can be generated effectively with a single sketch. Our method outperforms existing ones using other machine learning structures such as the variational autoencoder.

show abstract

“…Chai et al (Chai and Hodgins, 2005) generate a hu-Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning man surface from a sparse input with a large motion database. Shen et al (Shen et al, 2018) map complex gestures to crowd movement for gesture-based crowd control. Shum et al (Shum et al, 2013) reconstruct noisy human motion captured by Kinect.…”

Section: Machine Learning For 3d Shape Reconstructionmentioning

confidence: 99%

“…Learning a global model from all cars for fine details is therefore highly ineffective. Motivated by the success of lazy learning in mesh processing (Chai and Hodgins, 2005;Shen et al, 2018;Ho et al, 2013), we propose to adapt lazy learning to reconstructed the details of the car shape. Unlike traditional machine learning approaches that generalize data in the whole database as a preprocess, lazy learning postpones the generalization to run-time (Chai and Hodgins, 2005).…”

Section: Lazy Learning For Fine Detailsmentioning

confidence: 99%

Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Nozawa

Shum

et al. 2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multi-view depth and mask images, which form a more effective representation comparing to 3D meshes, and can be effectively fused to generate a 3D car shape. Since global models like deep learning have limited capacity to reconstruct fine-detail features, we propose a local lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car shape with detailed features is created. Experimental results show that the system performs consistently to create highly realistic cars of substantially different shape and topology.

show abstract

Data‐Driven Crowd Motion Control With Multi‐Touch Gestures

Cited by 11 publications

References 46 publications

Human Trajectory Prediction via Neural Social Physics

Human Trajectory Prediction via Neural Social Physics

3D car shape reconstruction from a contour sketch using GAN and lazy learning

Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Contact Info

Product

Resources

About