Knowledge Transfer for Scene-Specific Motion Prediction

Ballan, Lamberto; Castaldo, Francesco; Alahi, Alexandre; Palmieri, F.; Savarese, Silvio

doi:10.1007/978-3-319-46448-0_42

Cited by 90 publications

(83 citation statements)

References 48 publications

(98 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…detect semantic regions and learn how these affect agent behavior (Kitani et al 2012;Rehder and Kloeden 2015). Such semantics enable knowledge transfer to new scenes too (Ballan et al 2016). In surveillance, agent models are also used to reason about intent (Bandyopadhyay et al 2013), i.e.…”

Section: Static Environment Cuesmentioning

confidence: 99%

Context-Based Path Prediction for Targets with Switching Dynamics

et al. 2018

View full text Add to dashboard Cite

Anticipating future situations from streaming sensor data is a key perception challenge for mobile robotics and automated vehicles. We address the problem of predicting the path of objects with multiple dynamic modes. The dynamics of such targets can be described by a Switching Linear Dynamical System (SLDS). However, predictions from this probabilistic model cannot anticipate when a change in dynamic mode will occur. We propose to extract various types of cues with computer vision to provide context on the target's behavior, and incorporate these in a Dynamic Bayesian Network (DBN). The DBN extends the SLDS by conditioning the mode transition probabilities on additional context states. We describe efficient online inference in this DBN for probabilistic path prediction, accounting for uncertainty in both measurements and target behavior. Our approach is illustrated on two scenarios in the Intelligent Vehicles domain concerning pedestrians and cyclists, so-called Vulnerable Road Users (VRUs). Here, context cues include the static environment of the VRU, its dynamic environment, and its observed actions. Experiments using stereo vision data from a moving vehicle demonstrate that the proposed approach results in more accurate path prediction than SLDS at the relevant short time horizon (1 s). It slightly outperforms a computationally more demanding state-of-the-art method.

show abstract

Section: Static Environment Cuesmentioning

confidence: 99%

Context-Based Path Prediction for Targets with Switching Dynamics

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Recently, Kim et al [12], train a separate recurrent network, one for each future time step, to predict the location of nearby cars. Ballan et al [3] introduce a dynamic Bayesian network to model motion dependencies from previously seen patterns and apply them to unseen scenes by transferring the knowledge between similar settings. In an interesting work, a variational auto-encoders is used by Lee et al [14] to learn static scene context (and agents in a small neighborhood) and rank the generated trajectories accordingly.…”

Section: Related Workmentioning

confidence: 99%

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Sadeghian

Kosaraju

Sadeghian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

821

798

View full text Add to dashboard Cite

This paper addresses the problem of path prediction for multiple interacting agents in a scene, which is a crucial step for many autonomous platforms such as self-driving cars and social robots. We present SoPhie; an interpretable framework based on Generative Adversarial Network (GAN), which leverages two sources of information, the path history of all the agents in a scene, and the scene context information, using images of the scene. To predict a future path for an agent, both physical and social information must be leveraged. Previous work has not been successful to jointly model physical and social interactions. Our approach blends a social attention mechanism with a physical attention that helps the model to learn where to look in a large scene and extract the most salient parts of the image relevant to the path. Whereas, the social attention component aggregates information across the different agent interactions and extracts the most important trajectory information from the surrounding neighbors. SoPhie also takes advantage of GAN to generates more realistic samples and to capture the uncertain nature of the future paths by modeling its distribution. All these mechanisms enable our approach to predict socially and physically plausible paths for the agents and to achieve state-of-the-art performance on several different trajectory forecasting benchmarks.

show abstract

“…[3] introduces attractions towards static objects, such as artworks, which deflect straight paths in several scenarios (e.g., museums, galleries) but the approach is limited to a reduced number of static objects. [2] proposes a Bayesian framework based on previously observed motions to infer unobserved paths and for transferring learned motions to unobserved scenes. Similarly, in [6] circular distributions model dynamics and semantics for long-term trajectory predictions.…”

Section: Related Workmentioning

confidence: 99%

Social and Scene-Aware Trajectory Prediction in Crowded Spaces

Lisotto

Coscia

Ballan

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

Mimicking human ability to forecast future positions or interpret complex interactions in urban scenarios, such as streets, shopping malls or squares, is essential to develop socially compliant robots or self-driving cars. Autonomous systems may gain advantage on anticipating human motion to avoid collisions or to naturally behave alongside people. To foresee plausible trajectories, we construct an LSTM (long short-term memory)-based model considering three fundamental factors: people interactions, past observations in terms of previously crossed areas and semantics of surrounding space. Our model encompasses several pooling mechanisms to join the above elements defining multiple tensors, namely social, navigation and semantic tensors. The network is tested in unstructured environments where complex paths emerge according to both internal (intentions) and external (other people, not accessible areas) motivations. As demonstrated, modeling paths unaware of social interactions or context information, is insufficient to correctly predict future positions. Experimental results corroborate the effectiveness of the proposed framework in comparison to LSTM-based models for human path prediction.

show abstract

Knowledge Transfer for Scene-Specific Motion Prediction

Cited by 90 publications

References 48 publications

Context-Based Path Prediction for Targets with Switching Dynamics

Context-Based Path Prediction for Targets with Switching Dynamics

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Social and Scene-Aware Trajectory Prediction in Crowded Spaces

Contact Info

Product

Resources

About