Learning Neural Event Functions for Ordinary Differential Equations

Chen, Ricky T. Q.; Amos, Brandon; Nickel, Maximilian

doi:10.48550/arxiv.2011.03902

Cited by 16 publications

(30 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The closest work to ours is (Chen et al, 2020), in which neural event functions are introduced in neural ODE solvers to enable the learning of termination criteria. However, the design of such event function requires prior knowledge of the dynamical system.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

STRODE: Stochastic Boundary Ordinary Differential Equation

Huang,

Liu,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Perception of time from sequentially acquired sensory inputs is rooted in everyday behaviors of individual organisms. Yet, most algorithms for time-series modeling fail to learn dynamics of random event timings directly from visual or audio inputs, requiring timing annotations during training that are usually unavailable for real-world applications. For instance, neuroscience perspectives on postdiction imply that there exist variable temporal ranges within which the incoming sensory inputs can affect the earlier perception, but such temporal ranges are mostly unannotated for real applications such as automatic speech recognition (ASR). In this paper, we present a probabilistic ordinary differential equation (ODE), called STochastic boundaRy ODE (STRODE 1 ), that learns both the timings and the dynamics of time series data without requiring any timing annotations during training. STRODE allows the usage of differential equations to sample from the posterior point processes, efficiently and analytically. We further provide theoretical guarantees on the learning of STRODE. Our empirical results show that our approach successfully infers event timings of time series data. Our method achieves competitive or superior performances compared to existing state-of-the-art methods for both synthetic and real-world datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In contrast, our model adopts a regenerative point process as prior, whose parameters are learned from data itself. Furthermore, (Chen et al, 2020) focuses on solving IVPs, while our approach generalizes neural ODE in handling a special type of boundary value problem with random boundary conditions.…”

Section: Related Workmentioning

confidence: 99%

STRODE: Stochastic Boundary Ordinary Differential Equation

Huang,

Liu,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Simulating stochastic events While the event function approach appears to be limited to the deterministic setting, it also subsumes stochastic events whose aleatoric uncertainty is encoded by an intensity λ(t|H t ) [1], [13]. Without loss of generality let us consider a single intensity function which is henceforth denoted as λ * t := λ(t|H t ).…”

Section: Event Handling For Hybrid Systemsmentioning

confidence: 99%

Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions

Poli¹,

Massaroli²,

Scimeca³

et al. 2021

Preprint

View full text Add to dashboard Cite

Effective control and prediction of dynamical systems often require appropriate handling of continuous-time and discrete, event-triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi-modal continuous-time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi-mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number of modes and intermodal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations and self-supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end-to-end learning of hierarchical robot controllers.

show abstract

“…Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering such as predicting future movements of planets in physics, protein structure prediction [30], evolution of fluid flow [27] and many other applications [23]. The recently proposed neural ordinary differential equations (Neural ODEs) [1], a deep learning model integrated with differential equations shows great promise in scientific field [11,20,31,17,2]. The continuous nature of NODEs and their differential equation structure of the hypothesis have made them particularly suitable for learning the dynamics of complex physical systems.…”

Section: Introductionmentioning

confidence: 99%

“…MSE (interpolation MSE) is computed on time range [0, 1] year while Ex. MSE (extrapolation MSE) is computed on time range[1,2] year.…”

mentioning

confidence: 99%

Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics

Gong¹,

Meng²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering. Neural Ordinary Differential Equation (NODE), a deep learning model integrated with differential equations, learns the dynamics directly from the samples on the trajectory and shows great promise in the scientific field. However, the training of NODE highly depends on the numerical solver, which can amplify numerical noise and be unstable, especially for ill-conditioned dynamical systems. In this paper, to reduce the reliance on the numerical solver, we propose to enhance the supervised signal in learning dynamics. Specifically, beyond learning directly from the trajectory samples, we pre-train a neural differential operator (NDO) to output an estimation of the derivatives to serve as an additional supervised signal. The NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives. We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library. To leverage both the trajectory signal and the estimated derivatives from NDO, we propose an algorithm called NDO-NODE, in which the loss function contains two terms: the fitness on the true trajectory samples and the fitness on the estimated derivatives that are output by the pre-trained NDO. Experiments on various of dynamics show that our proposed NDO-NODE can consistently improve the forecasting accuracy.

show abstract

Learning Neural Event Functions for Ordinary Differential Equations

Cited by 16 publications

References 0 publications

STRODE: Stochastic Boundary Ordinary Differential Equation

STRODE: Stochastic Boundary Ordinary Differential Equation

Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions

Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics

Contact Info

Product

Resources

About