Parameter calibration with stochastic gradient descent for interacting particle systems driven by neural networks

Göttlich, Simone; Totzeck, Claudia

doi:10.1007/s00498-021-00309-8

Cited by 10 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…solutions of partial differential equations (42,43) or parameter estimation of multiagent models (44,45). Neural networks, and especially deep neural networks, are mathematically little understood, and their theoretical underpinnings are sparse and mainly restricted to shallow networks (networks with only one hidden layer) (46,47).…”

Section: Significancementioning

confidence: 99%

“…More recently, a promising new method has emerged in the form of artificial neural nets. Neural networks have of course prominently been used as powerful pattern-recognition devices and predictive models ( 41 ), but, as they become more and more accessible to the scientific community at large, researchers are beginning to apply their computational capabilities across the mathematical disciplines, including in fields heretofore dominated by more classical methods: Examples include finding solutions of partial differential equations ( 42 , 43 ) or parameter estimation of multiagent models ( 44 , 45 ). Neural networks, and especially deep neural networks, are mathematically little understood, and their theoretical underpinnings are sparse and mainly restricted to shallow networks (networks with only one hidden layer) ( 46 , 47 ).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Neural parameter calibration for large-scale multiagent models

Gaskin

Pavliotis

Girolami

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Computational models have become a powerful tool in the quantitative sciences to understand the behavior of complex systems that evolve in time. However, they often contain a potentially large number of free parameters whose values cannot be obtained from theory but need to be inferred from data. This is especially the case for models in the social sciences, economics, or computational epidemiology. Yet, many current parameter estimation methods are mathematically involved and computationally slow to run. In this paper, we present a computationally simple and fast method to retrieve accurate probability densities for model parameters using neural differential equations. We present a pipeline comprising multiagent models acting as forward solvers for systems of ordinary or stochastic differential equations and a neural network to then extract parameters from the data generated by the model. The two combined create a powerful tool that can quickly estimate densities on model parameters, even for very large systems. We demonstrate the method on synthetic time series data of the SIR model of the spread of infection and perform an in-depth analysis of the Harris–Wilson model of economic activity on a network, representing a nonconvex problem. For the latter, we apply our method both to synthetic data and to data of economic activity across Greater London. We find that our method calibrates the model orders of magnitude more accurately than a previous study of the same dataset using classical techniques, while running between 195 and 390 times faster.

show abstract

Section: Significancementioning

confidence: 99%

mentioning

confidence: 99%

Neural parameter calibration for large-scale multiagent models

Gaskin

Pavliotis

Girolami

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…In the context of calibration, the global maximum of the objective function is sought such that the model optimally matches the computational constraints. The CMA-ES algorithm, as an evolution strategy, is more suited to find such a global maximum compared to gradient-based approaches which are more likely to converge to local maxima, such as gradient descent (Galtier and Wainrib, 2013) or stochastic gradient descent (Göttlich and Totzeck, 2021). Figure 4 shows an example of CMA-ES maximization starting from an initial guess at iteration 0 until converging to one of two global maxima at iteration 20.…”

Section: Objective Function Maximization Illustrationmentioning

confidence: 99%

Integration of heterogeneous biological data in multiscale mechanistic model calibration: application to lung adenocarcinoma

Palgen¹,

Perrillat-Mercerot²,

Ceres³

et al. 2022

Preprint

View full text Add to dashboard Cite

Mechanistic models are built using knowledge as the primary information source, with well-established biological and physical laws determining the causal relationships within the model. Once the causal structure of the model is determined, parameters must be defined in order to accurately reproduce relevant data. Determining parameters and their values is particularly challenging in the case of models of pathophysiology, for which data for calibration is sparse. Multiple data sources might be required, and data may not be in a uniform or desirable format. We describe a calibration strategy to overcome the challenges of scarcity and heterogeneity of calibration data. Our strategy focuses on parameters where initial values cannot be easily derived from the literature and where final values are estimated via calibration with constraints set by relevant data. When combined with a covariance matrix adaptation evolution strategy (CMA-ES), this step-by-step approach can be applied to a wide range of biological models. We describe a stepwise, integrative and iterative approach to multiscale mechanistic model calibration, and provide an example of calibrating a pathophysiological lung adenocarcinoma model. Using the approach described here we illustrate the successful calibration of a complex knowledge-based mechanistic model using only the limited heterogeneous datasets publicly available in the literature.

show abstract

“…In general, this is done by using an training algorithm like the back-propagation algorithm [241]. The aim of this training algorithm is to adjust the network settings in a way, that minimizes the given cost function [242]. Common cost functions are the mean square error or the cross entropy.…”

Section: A Technically Oriented Comparisonmentioning

confidence: 99%

Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches

Korbmacher¹,

Tordeux²

2021

Preprint

View full text Add to dashboard Cite

In crowd scenarios, predicting trajectories of pedestrians is a complex and challenging task depending on many external factors. The topology of the scene and the interactions between the pedestrians are just some of them. Due to advancements in data-science and data collection technologies deep learning methods have recently become a research hotspot in numerous domains. Therefore, it is not surprising that more and more researchers apply these methods to predict trajectories of pedestrians. This paper compares these relatively new deep learning algorithms with classical knowledge-based models that are widely used to simulate pedestrian dynamics. It provides a comprehensive literature review of both approaches, explores technical and application oriented differences, and addresses open questions as well as future development directions. Our investigations point out that the pertinence of knowledge-based models to predict local trajectories is nowadays questionable because of the high accuracy of the deep learning algorithms. Nevertheless, the ability of deep-learning algorithms for large-scale simulation and the description of collective dynamics remains to be demonstrated. Furthermore, the comparison shows that the combination of both approaches (the hybrid approach) seems to be promising to overcome disadvantages like the missing explainability of the deep learning approach.

show abstract

Parameter calibration with stochastic gradient descent for interacting particle systems driven by neural networks

Cited by 10 publications

References 22 publications

Neural parameter calibration for large-scale multiagent models

Neural parameter calibration for large-scale multiagent models

Integration of heterogeneous biological data in multiscale mechanistic model calibration: application to lung adenocarcinoma

Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches

Contact Info

Product

Resources

About