On Automatic Differentiation and Algorithmic Linearization

Griewank, Andreas

doi:10.1590/0101-7438.2014.034.03.0621

Cited by 17 publications

(7 citation statements)

References 21 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We briefly describe AD as used in back-propagation. A detailed description of the working of automatic differentiation can be found in [34]- [36]. AD relies on the ability to decompose a program into a series of elementary operations (primitives) for which the derivatives are known and to which the chain rule can be applied [37].…”

Section: Pytorch Modellingmentioning

confidence: 99%

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Iqbal¹,

Behjat²,

Adlakha³

et al. 2022

Preprint

View full text Add to dashboard Cite

Physics-Infused Machine Learning (PIML) architectures aim at integrating machine learning with computationallyefficient, low-fidelity (partial) physics models, leading to improved generalizability, extrapolability, and robustness to noise, compared to pure data-driven approximation models. End-uses of PIML include, but are not limited to, model-based optimization and model-predictive control. Recently a new PIML architecture was reported by the same authors, known as Opportunistic Physics-mining Transfer Mapping Architecture or OPTMA, which transfers the original inputs into latent features using a transfer neural network; the partial physics model then uses the latent features to generate the final output that is as close as possible to the high-fidelity output. While gradient-free solvers and back-propagation with supervised learning (where optimum labels are pre-generated) have been used to train OPTMA, that approach remains computationally inefficient, and challenging to generalize across different problems or exploit state-of-theart machine learning architectures. This paper aims to alleviate these issues by infusing the partial physics model inside the neural network, as described via tensors in the popular machine learning framework, PyTorch. Such a description also naturally allows auto-differentiation of the partial physics model, thereby enabling the use of efficient back-propagation methods to train the transfer network. The benefits of the upgraded OPTMA architecture with Automatic Differentiation (OPTMA-Net) is demonstrated by applying it to the problem of modeling the sound pressure field created by a hovering unmanned aerial vehicle (UAV). Ground truth data for this problem has been obtained from an indoor UAV noise measurement setup. Here, the partial physics model is based on the interference of acoustic pressure waves generated by an arbitrary number of acoustic monopole sources. Case studies show that OPTMA-Net provides generalization performance close to, and extrapolation performance that is 4 times better than, those given by a pure data-driven model.

show abstract

Section: Pytorch Modellingmentioning

confidence: 99%

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Iqbal¹,

Behjat²,

Adlakha³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this paper, GAMC is equipped with the exponential schedule (15). Under schedule (15), GAMC and AM share similar convergence properties and complexity bounds asymptotically. Yet GAMC has faster mixing per step than AM due to exploitation of local geometric information in early phases of the chain.…”

mentioning

confidence: 99%

“…Yet GAMC has faster mixing per step than AM due to exploitation of local geometric information in early phases of the chain. The tuning parameter r in (15) regulates the frequency of geometric steps and therefore the ratio of mixing per step and computational cost per step. 4.5.…”

mentioning

confidence: 99%

“…Automatic differentiation (AD), a computationally driven research activity that has evolved since the mid 1950's, helps compute derivatives in a numerically exact way. Indeed, [15] has shown that AD is backward stable in the sense of [49]. Thus, small perturbations of the original function due to machine precision still yield accurate derivatives calculated via AD.…”

mentioning

confidence: 99%

“…Simulation study. In this simulation study, GAMC uses the exponential schedule (15) to switch randomly between the AM kernel specified via mixture (13) and the SMMALA kernel. GAMC is compared empirically against its AM and SM-MALA counterparts, as well as against MALA, in terms of mixing and cost per step via three examples.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Geometric adaptive Monte Carlo in random environment

Papamarkou¹,

Lindo²

2021

FoDS

View full text Add to dashboard Cite

Manifold Markov chain Monte Carlo algorithms have been introduced to sample more effectively from challenging target densities exhibiting multiple modes or strong correlations. Such algorithms exploit the local geometry of the parameter space, thus enabling chains to achieve a faster convergence rate when measured in number of steps. However, acquiring local geometric information can often increase computational complexity per step to the extent that sampling from high-dimensional targets becomes inefficient in terms of total computational time. This paper analyzes the computational complexity of manifold Langevin Monte Carlo and proposes a geometric adaptive Monte Carlo sampler aimed at balancing the benefits of exploiting local geometry with computational cost to achieve a high effective sample size for a given computational cost. The suggested sampler is a discrete-time stochastic process in random environment. The random environment allows to switch between local geometric and adaptive proposal kernels with the help of a schedule. An exponential schedule is put forward that enables more frequent use of geometric information in early transient phases of the chain, while saving computational time in late stationary phases. The average complexity can be manually set depending on the need for geometric exploitation posed by the underlying model.

show abstract

A higher-order error estimation framework for finite-volume CFD

Tyson

Roy

2019

Journal of Computational Physics

View full text Add to dashboard Cite

On Automatic Differentiation and Algorithmic Linearization

Cited by 17 publications

References 21 publications

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Geometric adaptive Monte Carlo in random environment

A higher-order error estimation framework for finite-volume CFD

Contact Info

Product

Resources

About