Learning to Simulate on Sparse Trajectory Data

Wei, Hua; Chen, Chacha; Liu, Chang; Zheng, Guopei; Li, Zhenhui

doi:10.1007/978-3-030-67667-4_32

Cited by 7 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Agarwal et al [22] uses simulation as an intermediate step to learn personalized policies in a data-sparse regime with heterogeneous users, where they only observe a single trajectory per user. Wei et al [24] proposes a framework for simulating in a data-sparse setting by using imitation learning to better interpolate traffic trajectories in an autonomous driving setting. In contrast, in PCS, the simulator is used as a crucial tool for using the framework to design, compare, and evaluate RL algorithm candidates for use in a particular problem setting.…”

Section: Simulation Environments In Reinforcement Learningmentioning

confidence: 99%

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines

et al. 2022

View full text Add to dashboard Cite

Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (predictability, computability, stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning to the design of RL algorithms for the digital interventions setting. Furthermore, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We show how we used the PCS framework to design an RL algorithm for Oralytics, a mobile health study aiming to improve users’ tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

show abstract

Section: Simulation Environments In Reinforcement Learningmentioning

confidence: 99%

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines

et al. 2022

View full text Add to dashboard Cite

show abstract

“…• GAIL considers the actions of the agents explicitly by learning the decision policy with generative adversarial networks (Ho and Ermon 2016;Song et al 2018;Zheng, Liu et al 2020;Wei et al 2020). Different from our proposed model, GAIL uses feed-forward networks without the system dynamics.…”

Section: Baselinesmentioning

confidence: 99%

How Do We Move: Modeling Human Movement with System Dynamics

Wei

Liang

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

Modeling how human moves in the space is useful for policy-making in transportation, public safety, and public health. The human movements can be viewed as a dynamic process that human transits between states (e.g., locations) over time. In the human world where intelligent agents like humans or vehicles with human drivers play an important role, the states of agents mostly describe human activities, and the state transition is influenced by both the human decisions and physical constraints from the real-world system (e.g., agents need to spend time to move over a certain distance). Therefore, the modeling of state transition should include the modeling of the agent's decision process and the physical system dynamics. In this paper, we propose MoveSD to model state transition in human movement from a novel perspective, by learning the decision model and integrating the system dynamics. MoveSD learns the human movement with Generative Adversarial Imitation Learning and integrates the stochastic constraints from system dynamics in the learning process. To the best of our knowledge, we are the first to learn to model the state transition of moving agents with system dynamics. In extensive experiments on real-world datasets, we demonstrate that the proposed method can generate trajectories similar to real-world ones, and outperform the state-of-the-art methods in predicting the next location and generating long-term future trajectories.

show abstract

“…In the specialized setting of traffic simulation, the issue of sparse sampling was approached by merging the tasks of imitation learning, and interpolation under a single generative adversarial network [63]. Moreover, the Ensemble-SINDy method learns noisy chaotic dynamics by averaging over a family of models formed using bootstrapped samples [17].…”

Section: Introductionmentioning

confidence: 99%

Learning Dynamical Systems From Invariant Measures

Botvinick-Greenhouse¹,

Martin²,

Yang³

2023

Preprint

View full text Add to dashboard Cite

We extend the methodology in [65] to learn autonomous continuous-time dynamical systems from invariant measures. We assume that our data accurately describes the dynamics' asymptotic statistics but that the available time history of observations is insufficient for approximating the Lagrangian velocity. Therefore, invariant measures are treated as the inference data and velocity learning is reformulated as a data-fitting, PDE-constrained optimization problem in which the stationary distributional solution to the Fokker-Planck equation is used as a differentiable surrogate forward model. We consider velocity parameterizations based upon global polynomials, piecewise polynomials, and fully connected neural networks, as well as various objective functions to compare synthetic and reference invariant measures. We utilize the adjoint-state method together with the backpropagation technique to efficiently perform gradient-based parameter identification. Numerical results for the Van der Pol oscillator and Lorenz-63 system, together with real-world applications to Hall-effect thruster dynamics and temperature prediction, are presented to demonstrate the effectiveness of the proposed approach.

show abstract

Learning to Simulate on Sparse Trajectory Data

Cited by 7 publications

References 19 publications

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines

How Do We Move: Modeling Human Movement with System Dynamics

Learning Dynamical Systems From Invariant Measures

Contact Info

Product

Resources

About