On Different Facets of Regularization Theory

Chen, Zhe; Haykin, S.

doi:10.1162/089976602760805296

Cited by 113 publications

(89 citation statements)

References 88 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Usually these measures are used to build bounds driving model selection, and may serve as regularization terms in a structural risk minimization approach. An overview of these and other regularization related topics is given in Mulier 2007, Chen andHaykin 2002). Less work has been done on measuring the behavioral of a model directly.…”

Section: Related Workmentioning

confidence: 99%

An alternative approach to avoid overfitting for surrogate models

Nguyen¹,

Couckuyt²,

Knockaert³

et al. 2011

Proceedings of the 2011 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

Surrogate models are data-driven models used to accurately mimic the complex behavior of a system. They are often used to approximate computationally expensive simulation code in order to speed up the exploration of design spaces. A crucial step in the building of surrogate models is finding a good set hyperparameters, which determine the behavior of the model. This is especially important when dealing with sparse data, as the models are in that case more prone to overfitting. Cross-validation is often used to optimize the hyperparameters of surrogate models, however it is computationally expensive and can still lead to overfitting or other erratic model behavior. This paper introduces a new auxiliary measure for the optimization of the hyperparameters of surrogate models which, when used in conjunction with a cheap accuracy measure, is fast and effective at avoiding unexplained model behavior.

show abstract

Section: Related Workmentioning

confidence: 99%

An alternative approach to avoid overfitting for surrogate models

Nguyen¹,

Couckuyt²,

Knockaert³

et al. 2011

Proceedings of the 2011 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

show abstract

“…The adopted Tikhonov regularization framework [34][35][36][37] is one of the most common forms of regularization. It minimizes an energy function in an RKHS H to regularize a function f , and can be written as:…”

Section: Transformation Estimationmentioning

confidence: 99%

A Small UAV Based Multi-Temporal Image Registration for Dynamic Agricultural Terrace Monitoring

Wei

Han

et al. 2017

Remote Sensing

View full text Add to dashboard Cite

Terraces are the major land-use type of agriculture and support the main agricultural production in southeast and southwest China. However, due to smallholder farming, complex terrains, natural disasters and illegal land occupations, a light-weight and low cost dynamic monitoring of agricultural terraces has become a serious concern for smallholder production systems in the above area. In this work, we propose a small unmanned aerial vehicle (UAV) based multi-temporal image registration method that plays an important role in transforming multi-temporal images into one coordinate system and determines the effectiveness of the subsequent change detection for dynamic agricultural terrace monitoring. The proposed method consists of four steps: (i) guided image filtering based agricultural terrace image preprocessing, (ii) texture and geometric structure features extraction and combination, (iii) multi-feature guided point set registration, and (iv) feature points based image registration. We evaluated the performance of the proposed method by 20 pairs of aerial images captured from Longji and Yunhe terraces, China using a small UAV (the DJI Phantom 4 Pro), and also compared against four state-of-the-art methods where our method shows the best alignments in most cases.

show abstract

“…Over some time, learning tasks based on the ERM principle may be ill-posed in science and engineering areas because there may be infinite hypotheses that satisfy the constraints [37] and even slight perturbations to the parameters of the model lead to very different solutions [34]. The principle of minimizing the errors on a training dataset is not self-evident, and another inductive principle and algorithms with higher generalization ability are required.…”

Section: Structural Risk Function Minimizationmentioning

confidence: 99%

Addressing Complexities of Machine Learning in Big Data: Principles, Trends and Challenges from Systematical Perspectives

Wang¹,

Zhao²,

Huang³

et al. 2017

Preprint

View full text Add to dashboard Cite

The concept of 'big data' has been widely discussed, and its value has been illuminated throughout a variety of domains. To quickly mine potential values and alleviate the ever-increasing volume of information, machine learning is playing an increasingly important role and faces more challenges than ever. Because few studies exist regarding how to modify machine learning techniques to accommodate big data environments, we provide a comprehensive overview of the history of the evolution of big data, the foundations of machine learning, and the bottlenecks and trends of machine learning in the big data era. More specifically, based on learning principals, we discuss regularization to enhance generalization. The challenges of quality in big data are reduced to the curse of dimensionality, class imbalances, concept drift and label noise, and the underlying reasons and mainstream methodologies to address these challenges are introduced. Learning model development has been driven by domain specifics, dataset complexities, and the presence or absence of human involvement. In this paper, we propose a robust learning paradigm by aggregating the aforementioned factors. Over the next few decades, we believe that these perspectives will lead to novel ideas and encourage more studies aimed at incorporating knowledge and establishing data-driven learning systems that involve both data quality considerations and human interactions.

show abstract

On Different Facets of Regularization Theory

Cited by 113 publications

References 88 publications

An alternative approach to avoid overfitting for surrogate models

An alternative approach to avoid overfitting for surrogate models

A Small UAV Based Multi-Temporal Image Registration for Dynamic Agricultural Terrace Monitoring

Addressing Complexities of Machine Learning in Big Data: Principles, Trends and Challenges from Systematical Perspectives

Contact Info

Product

Resources

About