Optimal stopping and effective machine complexity in learning

Wang, Changfeng; Venkatesh, Santosh S.; Judd, J. Stephen

doi:10.1109/isit.1995.531518

Cited by 14 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The test set is generated separately, with 100 configurations per temperature. From the training data, we randomly select 10% for cross-validation, in order to decrease the chance of overfitting and to identify a definitive stopping point for training using early stopping [31][32][33].…”

Section: Methods and Resultsmentioning

confidence: 99%

Machine learning vortices at the Kosterlitz-Thouless transition

2018

View full text Add to dashboard Cite

Efficient and automated classification of phases from minimally processed data is one goal of machine learning in condensed matter and statistical physics. Supervised algorithms trained on raw samples of microstates can successfully detect conventional phase transitions via learning a bulk feature such as an order parameter. In this paper, we investigate whether neural networks can learn to classify phases based on topological defects. We address this question on the twodimensional classical XY model which exhibits a Kosterlitz-Thouless transition. We find significant feature engineering of the raw spin states is required to convincingly claim that features of the vortex configurations are responsible for learning the transition temperature. We further show a single-layer network does not correctly classify the phases of the XY model, while a convolutional network easily performs classification by learning the global magnetization. Finally, we design a deep network capable of learning vortices without feature engineering. We demonstrate the detection of vortices does not necessarily result in the best classification accuracy, especially for lattices of less than approximately 1000 spins. For larger systems, it remains a difficult task to learn vortices.I.

show abstract

Section: Methods and Resultsmentioning

confidence: 99%

Machine learning vortices at the Kosterlitz-Thouless transition

2018

View full text Add to dashboard Cite

show abstract

“…An appropriate regularization parameter can be selected by cross-validation [100,114,115]. The dataset is randomly divided into a training set and a validation (test) set, with the major portion of the dataset included in the training set and the remaining in the validation set.…”

Section: Selection Of Regularization Parametermentioning

confidence: 99%

Perceptron: Learning, Generalization, Model Selection, Fault Tolerance, and Role in the Deep Learning Era

Leung

Mow

et al. 2022

Mathematics

View full text Add to dashboard Cite

The single-layer perceptron, introduced by Rosenblatt in 1958, is one of the earliest and simplest neural network models. However, it is incapable of classifying linearly inseparable patterns. A new era of neural network research started in 1986, when the backpropagation (BP) algorithm was rediscovered for training the multilayer perceptron (MLP) model. An MLP with a large number of hidden nodes can function as a universal approximator. To date, the MLP model is the most fundamental and important neural network model. It is also the most investigated neural network model. Even in this AI or deep learning era, the MLP is still among the few most investigated and used neural network models. Numerous new results have been obtained in the past three decades. This survey paper gives a comprehensive and state-of-the-art introduction to the perceptron model, with emphasis on learning, generalization, model selection and fault tolerance. The role of the perceptron model in the deep learning era is also described. This paper provides a concluding survey of perceptron learning, and it covers all the major achievements in the past seven decades. It also serves a tutorial for perceptron learning.

show abstract

“…To obtain the optimum numbers of delay taps and neurons in the hidden layer, several configurations were trained using various values of such parameters. The employed training algorithm was based on the Levenberg-Marquardt algorithm [55], and the early stopping method was utilized to stop the training [56]. Of the 10,800 total samples, 70% were used for training, 15% for validation, and the remaining 15% for evaluation.…”

Section: Ann-based Submodelmentioning

confidence: 99%