The Lack of A Priori Distinctions Between Learning Algorithms

Wolpert, David H.

doi:10.1162/neco.1996.8.7.1341

Cited by 1,285 publications

(697 citation statements)

References 5 publications

Supporting

Mentioning

617

Contrasting

Unclassified

Order By: Relevance

“…As the "No Free Lunch" states, there is no perfect algorithm [Wolpert 1996]. As with all modeling situations it is important to find the right tool for the job.…”

Section: Resultsmentioning

confidence: 99%

Random Forests for Genetic Association Studies

Goldstein

Polley²,

Briggs

2011

Statistical Applications in Genetics and Molecular Biology

202

172

View full text Add to dashboard Cite

The Random Forests (RF) algorithm has become a commonly used machine learning algorithm for genetic association studies. It is well suited for genetic applications since it is both computationally efficient and models genetic causal mechanisms well. With its growing ubiquity, there has been inconsistent and less than optimal use of RF in the literature. The purpose of this review is to breakdown the theoretical and statistical basis of RF so that practitioners are able to apply it in their work. An emphasis is placed on showing how the various components contribute to bias and variance, as well as discussing variable importance measures. Applications specific to genetic studies are highlighted. To provide context, RF is compared to other commonly used machine learning algorithms.

show abstract

“…As the "No Free Lunch" states, there is no perfect algorithm [Wolpert 1996]. As with all modeling situations it is important to find the right tool for the job.…”

Section: Resultsmentioning

confidence: 99%

Random Forests for Genetic Association Studies

Goldstein

Polley²,

Briggs

2011

Statistical Applications in Genetics and Molecular Biology

202

172

View full text Add to dashboard Cite

show abstract

“…By general-purpose or special-purpose we want to distinguish methods that apply to a large class versus a tiny class of tasks. Keeping in mind that there exists no completely universal statistical learning algorithm (Wolpert, 1996), it suffices that such broadly applicable generalization principles be relevant to the type of learning tasks that we care about, such as those solved by humans and animals.…”

Section: What Is Neededmentioning

confidence: 99%

On the challenge of learning complex functions

Bengio

2007

Progress in Brain Research

View full text Add to dashboard Cite

A common goal of computational neuroscience and of artificial intelligence research based on statistical learning algorithms is the discovery and understanding of computational principles that could explain what we consider adaptive intelligence, in animals as well as in machines. This chapter focuses on what is required for the learning of complex behaviors. We believe it involves the learning of highly varying functions, in a mathematical sense. We bring forward two types of arguments which convey the message that many currently popular machine learning approaches to learning flexible functions have fundamental limitations that render them inappropriate for learning highly varying functions. The first issue concerns the representation of such functions with what we call shallow model architectures. We discuss limitations of shallow architectures, such as so-called kernel machines, boosting algorithms, and one-hidden-layer artificial neural networks. The second issue is more focused and concerns kernel machines with a local kernel (the type used most often in practice), that act like a collection of template matching units. We present mathematical results on such computational architectures showing that they have a limitation similar to those already proved for older non-parametric methods, and connected to the so-called curse of dimensionality. Though it has long been believed that efficient learning in deep architectures is difficult, recently proposed computational principles for learning in deep architectures may offer a breakthrough.

show abstract

“…In addition to reporting and discussing the tabularized results, a Friedman test is conducted with a Nemenyi post hoc test [70] to rank the methods and describe their critical differences under all metric and problem type conditions. This is done to give some measure of generalizability [71,72] (commonly done when evaluating multiple classifiers over multiple problem instances), although problem instances and experimental variations are not exhaustive.…”

Section: Segment Quality Comparison and Methods Rankingmentioning

confidence: 99%

Classifier Directed Data Hybridization for Geographic Sample Supervised Segment Generation

Fourie

Schöepfer

2014

Remote Sensing

View full text Add to dashboard Cite

Quality segment generation is a well-known challenge and research objective within Geographic Object-based Image Analysis (GEOBIA). Although methodological avenues within GEOBIA are diverse, segmentation commonly plays a central role in most approaches, influencing and being influenced by surrounding processes. A general approach using supervised quality measures, specifically user provided reference segments, suggest casting the parameters of a given segmentation algorithm as a multidimensional search problem. In such a sample supervised segment generation approach, spatial metrics observing the user provided reference segments may drive the search process. The search is commonly performed by metaheuristics. A novel sample supervised segment generation approach is presented in this work, where the spectral content of provided reference segments is queried. A one-class classification process using spectral information from inside the provided reference segments is used to generate a probability image, which in turn is employed to direct a hybridization of the original input imagery. Segmentation is performed on such a hybrid image. These processes are adjustable, interdependent and form a part of the search problem. Results are presented detailing the performances of four method variants compared to the generic sample supervised segment generation approach, under various conditions in terms of resultant segment quality, required computing time and search process characteristics. Multiple metrics, metaheuristics and segmentation algorithms are tested with this approach. Using the spectral data contained within user OPEN ACCESS Remote Sens. 2014, 6 11853 provided reference segments to tailor the output generally improves the results in the investigated problem contexts, but at the expense of additional required computing time.

show abstract

The Lack of A Priori Distinctions Between Learning Algorithms

Cited by 1,285 publications

References 5 publications

Random Forests for Genetic Association Studies

Random Forests for Genetic Association Studies

On the challenge of learning complex functions

Classifier Directed Data Hybridization for Geographic Sample Supervised Segment Generation

Contact Info

Product

Resources

About