2017
DOI: 10.1080/01621459.2016.1195744
|View full text |Cite
|
Sign up to set email alerts
|

Generalized Additive Models for Gigadata: Modeling the U.K. Black Smoke Network Daily Data

Abstract: We develop scalable methods for fitting penalized regression spline based generalized additive models with of the order of 10 4 coefficients to up to 10 8 data. Computational feasibility rests on: (i) a new iteration scheme for estimation of model coefficients and smoothing parameters, avoiding poorly scaling matrix operations; (ii) parallelization of the iteration's pivoted block Cholesky and basic matrix operations; (iii) the marginal discretization of model covariates to reduce memory footprint, with effici… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

1
132
0

Year Published

2017
2017
2023
2023

Publication Types

Select...
5
2
1

Relationship

0
8

Authors

Journals

citations
Cited by 160 publications
(150 citation statements)
references
References 53 publications
1
132
0
Order By: Relevance
“…), and (3) new dimension reduction and estimation techniques facilitate efficient implementation for large data sets (Wood et al . , ). For comparison with our mechanistic Bayesian hierarchical model, we implemented a GAM that can be formally written as follows:yiBernoullipigfalse(pifalse)=boldxiβ+normalηt+normalηnormals,where, as before, yi is equal to 1 if the i th deer is CWD‐positive and 0 otherwise.…”
Section: Methodsmentioning
confidence: 99%
See 1 more Smart Citation
“…), and (3) new dimension reduction and estimation techniques facilitate efficient implementation for large data sets (Wood et al . , ). For comparison with our mechanistic Bayesian hierarchical model, we implemented a GAM that can be formally written as follows:yiBernoullipigfalse(pifalse)=boldxiβ+normalηt+normalηnormals,where, as before, yi is equal to 1 if the i th deer is CWD‐positive and 0 otherwise.…”
Section: Methodsmentioning
confidence: 99%
“…Guisan et al 2002;Wood & Augustin 2002). We compared a GAM to our hierarchical model because (1) the linear structure of the GAM is easy to interpret for statistical inference, (2) basis functions can be used to explicitly model the spatial and temporal process (Hefley et al 2017b), and (3) new dimension reduction and estimation techniques facilitate efficient implementation for large data sets (Wood et al 2015(Wood et al , 2017. For comparison with our mechanistic Bayesian hierarchical model, we implemented a GAM that can be formally written as follows:…”
Section: Comparison With Generalised Additive Modelsmentioning
confidence: 99%
“…Thus, currently, the high computational cost of fitting such large models precludes us from taking advantage of mixed models. Nevertheless, recent progress with similarly large models (Wood, Li, Shaddick, & Augustin, 2017) shows that the combination of LMMs with deconvolution modeling might be computationally feasible in future implementations.…”
Section: Outlook: Integration With Linear Mixed Modelsmentioning
confidence: 99%
“…In the long term, it will be promising to integrate deconvolution with mixed-effects modeling (Ehinger, 2019), but this will require large computational resources (because the EEG data of all participants has to be fitted simultaneously) as well as new algorithms for estimating sparse mixed-effects models (e.g. Wood, Li, Shaddick, & Augustin, 2017).…”
Section: Integrating Deconvolution With Linear Mixed Modelsmentioning
confidence: 99%