The Effect of Splitting of Raw Data into Training and Test Subsets on the Accuracy of Predicting Spatial Distribution by a Multilayer Perceptron

Baglaeva, Elena; Sergeev, Alexander; Shichkin, Andrey; Buevich, Alexander

doi:10.1007/s11004-019-09813-9

“…Motivated by the necessity to leverage non-i.i.d. samples in practical applications, and evidence that model's performance is affected by spatial correlation [16,17], the statistical community devised new error estimation methods using the spatial coordinates of the samples: h-block leave-one-out (1995). Developed for time-series data (i.e., data showing temporal dependency), the h-block leave-oneout method is based on the principle that stationary processes achieve a correlation length (the "h") after which the samples are not correlated.…”

Section: Introductionmentioning

confidence: 99%

Geostatistical Learning: Challenges and Opportunities

Hoffimann

¹

,

Zortea

²

,

Carvalho

³

et al. 2021

Front. Appl. Math. Stat.

View full text Add to dashboard Cite

Statistical learning theory provides the foundation to applied machine learning, and its various successful applications in computer vision, natural language processing and other scientific domains. The theory, however, does not take into account the unique challenges of performing statistical learning in geospatial settings. For instance, it is well known that model errors cannot be assumed to be independent and identically distributed in geospatial (a.k.a. regionalized) variables due to spatial correlation; and trends caused by geophysical processes lead to covariate shifts between the domain where the model was trained and the domain where it will be applied, which in turn harm the use of classical learning methodologies that rely on random samples of the data. In this work, we introduce the geostatistical (transfer) learning problem, and illustrate the challenges of learning from geospatial data by assessing widely-used methods for estimating generalization error of learning models, under covariate shift and spatial correlation. Experiments with synthetic Gaussian process data as well as with real data from geophysical surveys in New Zealand indicate that none of the methods are adequate for model selection in a geospatial context. We provide general guidelines regarding the choice of these methods in practice while new methods are being actively researched.

show abstract

“…Motivated by the necessity to leverage non-i.i.d. samples in practical applications, and evidence that model's performance is affected by spatial correlation [16,17], the statistical community devised new error estimation methods using the spatial coordinates of the samples: h-block leave-one-out (1995). Developed for time-series data (i.e.…”

Section: Introductionmentioning

confidence: 99%

Geostatistical Learning: Challenges and Opportunities

Hoffimann¹,

Zortea²,

Carvalho³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

Statistical learning theory provides the foundation to applied machine learning, and its various successful applications in computer vision, natural language processing and other scientific domains. The theory, however, does not take into account the unique challenges of performing statistical learning in geospatial settings. For instance, it is well known that model errors cannot be assumed to be independent and identically distributed in geospatial (a.k.a. regionalized) variables due to spatial correlation; and trends caused by geophysical processes lead to covariate shifts between the domain where the model was trained and the domain where it will be applied, which in turn harm the use of classical learning methodologies that rely on random samples of the data. In this work, we introduce the geostatistical (transfer) learning problem, and illustrate the challenges of learning from geospatial data by assessing widely-used methods for estimating generalization error of learning models, under covariate shift and spatial correlation. Experiments with synthetic Gaussian process data as well as with real data from geophysical surveys in New Zealand indicate that none of the methods are adequate for model selection in a geospatial context. We provide general guidelines regarding the choice of these methods in practice while new methods are being actively researched.

show abstract