laGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes inR

Abstract

Gaussian process (GP) regression models make for powerful predictors in out of sample exercises, but cubic runtimes for dense matrix decompositions severely limit the size of data -training and testing -on which they can be deployed. That means that in computer experiment, spatial/geo-physical, and machine learning contexts, GPs no longer enjoy privileged status as data sets continue to balloon in size. We discuss an implementation of local approximate Gaussian process models, in the laGP package for R, that offers a particular sparse-matrix remedy uniquely positioned to leverage modern parallel computing architectures. The laGP approach can be seen as an update on the spatial statistical method of local kriging neighborhoods. We briefly review the method, and provide extensive illustrations of the features in the package through worked-code examples. The appendix covers custom building options for symmetric multi-processor and graphical processing units, and built-in wrapper routines that automate distribution over a simple network of workstations.

Prediction for Big Data Through Kriging: Small Sequential and One-Shot Designs

“…Big data implies that N is so big that the computation of R 1 and jRj gives numerical problems. More precisely, these computations require O(N 3 ) matrix decompositions; see Damianou (2015, p. 20), Gramacy (2016), Nickson et al (2015), and Van Stein et al (2017). Furthermore, the more space-…lling (or clustered) the design is, the higher the condition number of R is; see Lim et al (2017).…”

“…Furthermore, we assume that the simulation is run on a traditional personal computer; i.e., we do not consider parallel computing architectures; such architectures are considered in Gramacy (2016), , Guhaniyogi and Banerjee (2018), Guinness (2018), Gutiérrez de Ravé et al (2014), Morgan et al (2017), Van Stein et al (2017), and Xu et al (2016).…”

“…We notice that Gramacy (2016) also allows output that has noise with a small constant "intrinsic" variance; in future research we may investigate intrinsic noise with big variances that vary with the input combinations; also see Section 6. We use a frequentist approach (instead of a Bayesian approach, which is used in Angelino et al (2016), Damianou (2015), Guhaniyogi and Banerjee (2018), Gramacy (2016), Nickson et al (2015), and Pronzato and Rendas (2017)). An advantage of such a frequentist approach is that we do not need to specify a prior distribution for the Kriging (hyper)parameters; we …nd it hard to specify such a prior distribution.…”

“…We also refer to Bradley et al (2016), Chilès and Desassis (2018), Damianou (2015), Guinness (2018), Meng and Ng (2016), Nickson et al (2015), Sung et al (2016), Tzeng and Huang (2018), and Van Stein et al (2017). We, however, focus on the method that is detailed in Gramacy (2016) and Gramacy and Apley (2015), and compare our methods with Gramacy's method (Gramacy's method is also compared with a new method in Guhaniyogi and Banerjee (2018)). We observe that two NN variations are used in Gramacy (2016, Section 3.1) and Gramacy and Apley (2015, Fig.…”

Simulation Optimization Through Regression or Kriging Metamodels

“…Consequently, a multitude of factor combinations may be simulated (also see our comments on "expensive" and "cheap" simulations). Moreover, simulation is well-suited to "sequential" designs instead of "one shot" designs, because simulation experiments run on computers that typically produce output sequentially (apart from parallel computers; see this book and Gramacy (2015)), whereas agricultural experiments run during a single growing season. Altogether, many simulation analysts need a change of mindset.…”