Supervised Learning: Can it Escape its Local Minimum?

Werbos, Paul J.

doi:10.1007/978-1-4615-2696-4_13

Cited by 13 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Nnet mimics a feed-forward neural network that uses the backpropagation algorithm for training coupled with one hidden layer [86]. To calibrate the Nnet algorithm, there are three important parameters required, namely the size, decay, and maxit, which control the number of neurons in the hidden layer, the weight decay, and the maximum number of iterations, respectively.…”

Section: Deep Learning Classifiersmentioning

confidence: 99%

Climate-Based Regionalization and Inclusion of Spectral Indices for Enhancing Transboundary Land-Use/Cover Classification Using Deep Learning and Machine Learning

2021

View full text Add to dashboard Cite

Accurate land use and cover data are essential for effective land-use planning, hydrological modeling, and policy development. Since the Okavango Delta is a transboundary Ramsar site, managing natural resources within the Okavango Basin is undoubtedly a complex issue. It is often difficult to accurately map land use and cover using remote sensing in heterogeneous landscapes. This study investigates the combined value of climate-based regionalization and integration of spectral bands with spectral indices to enhance the accuracy of multi-temporal land use/cover classification using deep learning and machine learning approaches. Two experiments were set up, the first entailing the integration of spectral bands with spectral indices and the second involving the combined integration of spectral indices and climate-based regionalization based on Koppen–Geiger climate zones. Landsat 5 TM and Landsat 8 OLI images, machine learning classifiers (random forest and extreme gradient boosting), and deep learning (neural network and deep neural network) classifiers were used in this study. Supervised classification using a total of 5140 samples was conducted for the years 1996, 2004, 2013, and 2020. Average overall accuracy and Kappa coefficients were used to validate the results. The study found that the integration of spectral bands with indices improves the accuracy of land use/cover classification using machine learning and deep learning. Post-feature selection combinations yield higher accuracies in comparison to combinations of bands and indices. A combined integration of spectral indices with bands and climate-based regionalization did not significantly improve the accuracy of land use/cover classification consistently for all the classifiers (p < 0.05). However, post-feature selection combinations and climate-based regionalization significantly improved the accuracy for all classifiers investigated in this study. Findings of this study will improve the reliability of land use/cover monitoring in complex heterogeneous TDBs.

show abstract

Section: Deep Learning Classifiersmentioning

confidence: 99%

Climate-Based Regionalization and Inclusion of Spectral Indices for Enhancing Transboundary Land-Use/Cover Classification Using Deep Learning and Machine Learning

2021

View full text Add to dashboard Cite

show abstract

“…(1) Even in the "simple" testbed problem of supervised learning (learning a static mapping from an input vector X to a vector of targets or dependent variables Y), we need improved learning speed and generalization ability, exploiting concepts such as "syncretism" (section 2.2) and "simultaneous recurrence" [32]. Also see [34].…”

Section: The Search For True Intelligent Controlmentioning

confidence: 99%

<title>Stable adaptive control using new critic designs</title>

Werbos

1999

SPIE Proceedings

Self Cite

View full text Add to dashboard Cite

Classical adaptive control proves total-system stability for control of linear plants, but only for plants meeting very restrictive assumptions. Approximate Dynamic Programming (ADP) has the potential, in principle, to ensure stability without such tight restrictions. It also offers nonlinear and neural extensions for optimal control, with empirically supported links to what is seen in the brain. However, the relevant ADP methods in use today --TD, HDP, DHP, GDHP --and the Galerkin-based versions of these all have serious limitations when used here as parallel distributed real-time learning systems; either they do not possess quadratic unconditional stability (to be defined) or they lead to incorrect results in the stochastic case. (ADAC or Q-learning designs do not help.) After explaining these conclusions, this paper describes new ADP designs which overcome these limitations. It also addresses the Generalized Moving Target problem, a common family of static optimization problems, and describes a way to stabilize large-scale economic equilibrium models, such as the old long-term energy model of DOE.

show abstract

“…Finding a nonlinear adaptive system like a neural network with an appropriate structure is a nonlinear optimization problem, which can be complex and slow. One of the most widely used algorithms for training neural networks is backpropagation (BP) based on gradient descent [90], [103]. Although this simple approach has been successfully used to train neural networks for a wide range of applications, the algorithm can be notoriously slow when used to train a large network in solving a complex problem.…”

Section: Fast Training Algorithmsmentioning

confidence: 99%

Performance and efficiency: recent advances in supervised learning

1999

Proc. IEEE

View full text Add to dashboard Cite

This paper reviews recent advances in supervised learning with a focus on two most important issues: performance and efficiency. Performance addresses the generalization capability of a learning machine on randomly chosen samples that are not included in a training set. Efficiency deals with the complexity of a learning machine in both space and time. As these two issues are general to various learning machines and learning approaches, we focus on a special type of adaptive learning systems with a neural architecture. We discuss four types of learning approaches: training an individual model; combinations of several well-trained models; combinations of many weak models; and evolutionary computation of models. We explore advantages and weaknesses of each approach and their interrelations, and we pose open questions for possible future research.

show abstract

Supervised Learning: Can it Escape its Local Minimum?

Cited by 13 publications

References 6 publications

Climate-Based Regionalization and Inclusion of Spectral Indices for Enhancing Transboundary Land-Use/Cover Classification Using Deep Learning and Machine Learning

Climate-Based Regionalization and Inclusion of Spectral Indices for Enhancing Transboundary Land-Use/Cover Classification Using Deep Learning and Machine Learning

<title>Stable adaptive control using new critic designs</title>

Performance and efficiency: recent advances in supervised learning

Contact Info

Product

Resources

About