70 years of machine learning in geoscience in review

Abstract

This review gives an overview of the development of machine learning in geoscience. A thorough analysis of the codevelopments of machine learning applications throughout the last 70 years relates the recent enthusiasm for machine learning to developments in geoscience. I explore the shift of kriging toward a mainstream machine learning method and the historic application of neural networks in geoscience, following the general trend of machine learning enthusiasm through the decades. Furthermore, this chapter explores the shift from mathematical fundamentals and knowledge in software development toward skills in model validation, applied statistics, and integrated subject matter expertise. The review is interspersed with code examples to complement the theoretical foundations and illustrate model validation and machine learning explainability for science. The scope of this review includes various shallow machine learning methods, e.g., decision trees, random forests, support-vector machines, and Gaussian processes, as well as, deep neural networks, including feed-forward neural networks, convolutional neural networks, recurrent neural networks, and generative adversarial networks. Regarding geoscience, the review has a bias toward geophysics but aims to strike a balance with geochemistry, geostatistics, and geology, however, excludes remote sensing, as this would exceed the scope. In general, I aim to provide context for the recent enthusiasm surrounding deep learning with respect to research, hardware, and software developments that enable successful application of shallow and deep machine learning in all disciplines of Earth science.

Multiple imputation via chained equations for elastic well log imputation and prediction

“…There are many approaches to estimating missing values which range from simple mean or median estimates, to regression models. More recently machine learning algorithms have been gaining in popularity in the geosciences (Dramsch, 2020). The efficacy of any method depends upon the type of data being imputed, the quality and distribution of the known values and the complexity of the imputation model.…”

“…The recent explosion in data science methods and availability of large amounts of well log data present an opportunity to use a more automated statistics based approach which simplifies the imputation process, improving both accuracy and turnaround. The application of machine learning methods to well log imputation or prediction and with geosciences in general is not new (Dramsch, 2020). For example, commercial applications of earlier machine learning algorithms (artificial neural networks (ANN) and radial basis functions) have been used to predict logs from seismic data and attributes (Hampson et al, 2001;Russell et al, 2003) have existed for nearly 20 years.…”

Laboratory earthquake forecasting: A machine learning competition

“…The confluence of new ML algorithms, fast and inexpensive graphical processing units and tensor processing units, and the availability of massive, often continuous datasets has driven this revolution in data-driven analysis. This rapid expansion has seen application of existing and new ML tools to a suite of geoscientific problems ( 33 – 36 ) that span seismic wave detection and phase identification and location ( 34 , 37 – 45 ), geological formation identification ( 46 , 47 ), earthquake early warning ( 48 ), volcano monitoring ( 49 – 51 ), denoising Interferometric Synthetic Aperture Radar (InSAR) ( 50 , 52 , 53 ), tomographic imaging ( 54 – 57 ), reservoir characterization ( 58 – 60 ), and more. Of particular note is that, over the past 5 y, considerable effort has been devoted to using these approaches to characterize fault physics and forecast fault failure ( 1 – 3 , 13 , 35 , 61 – 63 ).…”

Automatic Fault Mapping in Remote Optical Images and Topographic Data With Deep Learning

“…Machine learning comprises a plethora of numerical methods that can learn from available data and make predictions in unseen data (process called generalization, e.g., Abu‐Mostafa et al., 2012; Goodfellow, Bengio, et al., 2016). Machine learning is not a new domain of research, with the first landmark works having emerged in the 50–60's (see Figure 1 in Dramsch (2020) and references therein). However, its demands for large quantities of data and computational resources required by the learning process have only been met over the last two decades or so (e.g., Bishop, 1995; Carbonell et al., 1983; Devilee et al., 1999; Ermini et al., 2005; Goodfellow, Bengio, et al., 2016; LeCun, Bengio, & Hinton, 2015; MacKay, 2003; Meier et al., 2007; Mjolsness & DeCoste, 2001; Van der Baan & Jutten, 2000).…”