Conditional Neural Processes

Garnelo, Marta; Rosenbaum, Dan T.; Maddison, Chris J.; Ramalho, Tiago; Saxton, David; Shanahan, Murray; Teh, Yee Whye; Rezende, Danilo Jimenez; Eslami, S. M. Ali

doi:10.48550/arxiv.1807.01613

Cited by 31 publications

(60 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider three aspects in our future work: i) we are going to apply NPBO in different scenarios, e.g., accelerating experiments in the physical science (Ermon 2020); ii) we will test the performance of variants of NP as the surrogate model, such as Conditional Neural Process (Garnelo et al 2018a) and Attentive Neural Process (Kim et al 2019); iii) acquisition function could also be replaced by NN to perform the trade-off strategy under Bayesian optimization framework.…”

Section: Discussionmentioning

confidence: 99%

Neural Process for Black-Box Model Optimization Under Bayesian Framework

Shangguan¹,

Lin²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

There are a large number of optimization problems in physical models where the relationships between model parameters and outputs are unknown or hard to track. These models are named as "black-box models" in general because they can only be viewed in terms of inputs and outputs, without knowledge of the internal workings. Optimizing the blackbox model parameters has become increasingly expensive and time consuming as they have become more complex. Hence, developing effective and efficient black-box model optimization algorithms has become an important task. One powerful algorithm to solve such problem is Bayesian optimization, which can effectively estimates the model parameters that lead to the best performance, and Gaussian Process (GP) has been one of the most widely used surrogate model in Bayesian optimization. However, the time complexity of GP scales cubically with respect to the number of observed model outputs, and GP does not scale well with large parameter dimension either. Consequently, it has been challenging for GP to optimize black-box models that need to query many observations and/or have many parameters. To overcome the drawbacks of GP, in this study, we propose a general Bayesian optimization algorithm that employs a Neural Process (NP) as the surrogate model to perform black-box model optimization, namely, Neural Process for Bayesian Optimization (NPBO). In order to validate the benefits of NPBO, we compare NPBO with four benchmark approaches on a power system parameter optimization problem and a series of seven benchmark Bayesian optimization problems. The results show that the proposed NPBO performs better than the other four benchmark approaches on the power system parameter optimization problem and competitively on the seven benchmark problems.

show abstract

Section: Discussionmentioning

confidence: 99%

Neural Process for Black-Box Model Optimization Under Bayesian Framework

Shangguan¹,

Lin²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…first ∼ 8 hours. For comparison, we also apply the Conditional Neural Process (CNP, Garnelo et al, 2018) to the cut light curves in order to test whether time-lag retrieval accuracy is improved (see Fig. 3 , right panels).…”

Section: Two Examples Of Lsst Opsim Runs Used In Our Lsst Agnmentioning

confidence: 99%

Photoreverberation mapping of quasars in the context of LSST observing strategies

Jankov,

Kovacevic,

Ilic

et al. 2021

Preprint

View full text Add to dashboard Cite

The upcoming photometric surveys, such as the Rubin Observatory's Legacy Survey of Space and Time (LSST) will monitor unprecedented number of active galactic nuclei (AGN) in a decade long campaign. Motivated by the science goals of LSST, which includes the harnessing of broadband light curves of AGN for photometric reverberation mapping (PhotoRM), we implement the existing formalism to estimate the lagged response of the emission line flux to the continuum variability using only mutli-band photometric light curves. We test the PhotoRM method on a set of 19 artificial light curves simulated using a stochastic model based on the Damped Random Walk process. These light curves are sampled using different observing strategies, including the two proposed by the LSST, in order to compare the accuracy of timelag retrieval based on different observing cadences. Additionally, we apply the same procedure for time-lag retrieval to the observed photometric light curves of NGC 4395, and compare our results to the existing literature.

show abstract

“…The GP-LSTM model indeed performs well in time series prediction, yet the computational efficiency of the model is limited by the highly intensive kernel functions. The conditional neural processes (CNPs) not only perform well when dealing with independent and identically distributed random variables [9], but also avoids training from scratch by extracting prior knowledge. Therefore, it can make predictions after observing only a small amount of data.…”

Section: Our Contribution: Novelties and Outlinementioning

confidence: 99%

PCNN: A physics-constrained neural network for multiphase flows

Zheng,

Huang,

Lin

2021

Preprint

View full text Add to dashboard Cite

The present study develops a physics-constrained neural network (PCNN) to predict sequential patterns and motions of multiphase flows (MPFs), which includes strong interactions among various fluid phases. To predict the order parameters, which locate individual phases, in the future time, the conditional neural processes and long short-term memory (CNP-LSTM) are applied to quickly infer the dynamics of the phases after encoding only a few observations. After that, the multiphase consistent and conservative boundedness mapping algorithm (MCBOM) is implemented to correct the order parameters predicted from CNP-LSTM in order to strictly satisfy the mass conservation, the summation of the volume fractions of the phases to be unity, the consistency of reduction, and the boundedness of the order parameters. Then, the density of the fluid mixture is updated from the corrected order parameters. Finally, the velocity in the future time is predicted by a physics-informed CNP-LSTM (PICNP-LSTM) where conservation of momentum is included in the loss function with the observed density and velocity as the inputs. The proposed PCNN for MPFs sequentially performs (CNP-LSTM)-(MCBOM)-(PICNP-LSTM), which avoids unphysical behaviors of the order parameters, accelerates the convergence, and requires fewer data to make predictions. Numerical experiments demonstrate that the proposed PCNN is capable of predicting MPFs effectively.

show abstract

Conditional Neural Processes

Cited by 31 publications

References 0 publications

Neural Process for Black-Box Model Optimization Under Bayesian Framework

Neural Process for Black-Box Model Optimization Under Bayesian Framework

Photoreverberation mapping of quasars in the context of LSST observing strategies

PCNN: A physics-constrained neural network for multiphase flows

Contact Info

Product

Resources

About