Gradient boosting machines, a tutorial

Natekin, Alexey; Knoll, Alois

doi:10.3389/fnbot.2013.00021

Cited by 2,081 publications

(1,282 citation statements)

References 51 publications

Supporting

Mentioning

1,100

Contrasting

Unclassified

Order By: Relevance

“…In gradient boosting modeling, more accurate estimate of the response variable is obtained through consecutively fitting new models in order to reduce the variance between the predicted and observed responses. The main idea of GBM is to learn the data to achieve maximum correlation with the negative gradient of the loss function (Natekin and Knoll 2013). For continuous response, the loss function can be Gaussian or Laplace functions.…”

Section: Generalized Boosted Regression Modelmentioning

confidence: 99%

“…For continuous response variable, the appropriate loss function is the squared-error L2 loss and its derivative represents the residual. So, the GBM can be applied based on the residual fitting (Natekin and Knoll 2013). The GBM procedure begins with assigning a differentiable loss function, which represents the variance between observed and predicted response factor y and starts with an initial model F, which can be averaging of y.…”

Section: Generalized Boosted Regression Modelmentioning

confidence: 99%

See 1 more Smart Citation

Integrating well log interpretations for lithofacies classification and permeability modeling through advanced machine learning algorithms

Al‐Mudhafar

2017

J Petrol Explor Prod Technol

135

View full text Add to dashboard Cite

In this paper, an integrated procedure was adopted to obtain accurate lithofacies classification to be incorporated with well log interpretations for a precise core permeability modeling. Probabilistic neural networks (PNNs) were employed to model lithofacies sequences as a function of well logging data in order to predict discrete lithofacies distribution at missing intervals. Then, the generalized boosted regression model (GBM) was used as to build a nonlinear relationship between core permeability, well logging data, and lithofacies. The well log interpretations that were considered for lithofacies classification and permeability modeling are neutron porosity, shale volume, and water saturation as a function of depth; however, the measured discrete lithofacies types are sand, shaly sand, and shale. Accurate lithofacies classification was achieved by the PNN as the total percent correct of the predicted discrete lithofacies was 95.81%. In GBM results, root-mean-square prediction error and adjusted R-square have incredible positive values, as there was an excellent matching between the measured and predicted core permeability. Additionally, the GBM model led to overcome the multicollinearity that was available between one pair of the predictors. The efficiency of boosted regression was demonstrated by the prediction matching of core permeability in comparison with the conventional multiple linear regression (MLR). GBM led to much more accurate permeability prediction than the MLR.

show abstract

Section: Generalized Boosted Regression Modelmentioning

confidence: 99%

Section: Generalized Boosted Regression Modelmentioning

confidence: 99%

Integrating well log interpretations for lithofacies classification and permeability modeling through advanced machine learning algorithms

Al‐Mudhafar

2017

J Petrol Explor Prod Technol

135

View full text Add to dashboard Cite

show abstract

“…This result is achieved by making the new decision tree maximally correlated with the negative gradient of the loss function. A detailed review of GB methods is given in Natekin & Knoll (2013) …”

Section: Gradient Tree Boostingmentioning

confidence: 99%

ASTErIsM: application of topometric clustering algorithms in automatic galaxy detection and classification

Tramacere

Paraficz

Dubath

et al. 2016

Mon. Not. R. Astron. Soc.

View full text Add to dashboard Cite

We present a study on galaxy detection and shape classification using topometric clustering algorithms. We first use the DBSCAN algorithm to extract, from CCD frames, groups of adjacent pixels with significant fluxes and we then apply the DENCLUE algorithm to separate the contributions of overlapping sources. The DENCLUE separation is based on the localization of pattern of local maxima, through an iterative algorithm which associates each pixel to the closest local maximum.Our main classification goal is to take apart elliptical from spiral galaxies. We introduce new sets of features derived from the computation of geometrical invariant moments of the pixel group shape and from the statistics of the spatial distribution of the DENCLUE local maxima patterns. Ellipticals are characterized by a single group of local maxima, related to the galaxy core, while spiral galaxies have additional ones related to segments of spiral arms. We use two different supervised ensemble classification algorithms, Random Forest, and Gradient Boosting. Using a sample of 24000 galaxies taken from the Galaxy Zoo 2 main sample with spectroscopic redshifts, and we test our classification against the Galaxy Zoo 2 catalog.We find that features extracted from our pipeline give on average an accuracy of 93%, when testing on a test set with a size of 20% of our full data set, with features deriving from the angular distribution of density attractor ranking at the top of the discrimination power.

show abstract

“…This technique is flexible enough to be used with different families of loss functions, and this choice is often influenced by the desired characteristics of the conditional distribution, such as robustness to outliers. Any arbitrary loss function can be plugged into the framework by specifying the loss function and the function to compute its negative gradient [17]. The squared 2 loss and the Laplacian…”

Section: Gradient Boosted Regression Treesmentioning

confidence: 99%

Identifying the Culprits Behind Network Congestion

Bhatelé

Titus

Thiagarajan

et al. 2015

2015 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

Abstract-Network congestion is one of the primary causes of performance degradation, performance variability and poor scaling in communication-heavy parallel applications. However, the causes and mechanisms of network congestion on modern interconnection networks are not well understood. We need new approaches to analyze, model and predict this critical behavior in order to improve the performance of large-scale parallel applications. This paper applies supervised learning algorithms, such as forests of extremely randomized trees and gradient boosted regression trees, to perform regression analysis on communication data and application execution time. Using data derived from multiple executions, we create models to predict the execution time of communication-heavy parallel applications. This analysis also identifies the features and associated hardware components that have the most impact on network congestion and in turn, on execution time. The ideas presented in this paper have wide applicability: predicting the execution time on a different number of cores, or different input datasets, or even for an unknown code, identifying the best configuration parameters for an application, and finding the root causes of network congestion on different architectures. I. MOTIVATION AND IMPACTNetwork congestion is widely recognized as one of the primary causes of performance degradation, performance variability, and poor scaling in communication-heavy applications running on supercomputers [5]. However, due to the complex nature of interconnection networks, as well as message injection and routing strategies, network congestion and its root causes for network resources and hardware components are not well understood. This makes the problem of mitigating and avoiding network congestion difficult. It also complicates the task of writing congestionavoiding and congestion-minimizing algorithms for communication and task mapping. Therefore, we need new approaches to understand and model network congestion in order to improve the performance of large-scale parallel applications.When a message is sent from one node to another, it is split into packets that pass through many resources and hardware components on the network. A packet starts in an injection FIFO on the source. It then passes through multiple network links and receive buffers on intermediate nodes before it finally lands in the reception FIFO on the destination. When shared by multiple packets, any or all of these network components can slow down individual flits, packets and messages. This paper aims to identify the hardware components that affect the performance of sending a message the most.Our approach is based on using supervised machine learning to build models that map from independent variables, representing different network hardware components, to a dependent variable -the execution time of the application. We only consider computationally balanced, communicationheavy parallel applications and, hence, focus on the communication fraction of the total executi...

show abstract

Gradient boosting machines, a tutorial

Cited by 2,081 publications

References 51 publications

Integrating well log interpretations for lithofacies classification and permeability modeling through advanced machine learning algorithms

Integrating well log interpretations for lithofacies classification and permeability modeling through advanced machine learning algorithms

ASTErIsM: application of topometric clustering algorithms in automatic galaxy detection and classification

Identifying the Culprits Behind Network Congestion

Contact Info

Product

Resources

About