Introduction to Statistical Learning Theory

Bousquet, Olivier; Boucheron, Stéphane; Lugosi, Gábor

doi:10.1007/978-3-540-28650-9_8

Cited by 346 publications

(368 citation statements)

References 67 publications

Supporting

Mentioning

355

Contrasting

Unclassified

Order By: Relevance

“…Develop a more effective multi-class classifier than its original design, which is exactly consistent with the theory that the generalization of any learning method depends on both the representation of data and exploitation of prior knowledge [17], consequently validate the prior structure of individual original classes for ECOC-based multi-class classification.…”

Section: Introductionsupporting

confidence: 63%

“…In the learning process, each "meta-class" will be treated as a single class, which brings a simple and common implementation of multi-class classification, but simultaneously results in the under-exploitation of already-provided structure knowledge in individual original classes. It is well-known that for any learning method, its generalization depends on both the representation of data and exploitation of prior knowledge for the current learning problem [17], as a result, for better generalization performance, we should explore as much prior knowledge as possible, let alone the knowledge provided already. In this paper, we present a methodology to show that utilizing such prior structure knowledge in implementing ECOCs can further strengthen the multi-class classification performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Can under-exploited structure of original-classes help ECOC-based multi-class classification?

2012

View full text Add to dashboard Cite

Section: Introductionsupporting

confidence: 63%

Section: Introductionmentioning

confidence: 99%

Can under-exploited structure of original-classes help ECOC-based multi-class classification?

2012

View full text Add to dashboard Cite

“…The classifier h : X → Y predicts the class label of a given input patternx asŷ = h(x). For details of the classifier training process, the reader is referred to Bousquet et al [2004]. …”

Section: Adaptive Classificationmentioning

confidence: 99%

A sequential sampling strategy for adaptive classification of computationally expensive data

Singh

Herten

Deschrijver

et al. 2016

Struct Multidisc Optim

View full text Add to dashboard Cite

Many real-world engineering problems can be represented and solved as classification problems. These problems are typically encountered in design optimization and use expensive data, since the data are often sourced from computationally expensive simulations. It is therefore crucial to solve the classification problem using as little data as possible. This necessitates an iterative classifier construction procedure beginning with a very small training set, which is supplemented in each iteration by a small batch of new data points. The sequential sampling algorithm selects locations in the input space and the simulator calculates the corresponding class label. This paper describes a sequential sampling algorithm for adaptive classification of expensive data.

show abstract

“…The concept of statistical learning theory [19,73,100,135,141] provides the mathematical framework for the theoretical analysis of machine learning techniques. Basis for such an analysis is an input space X ⊆ R d , an output space Y ⊆ R, and random variables (X, Y ) ∈ X × Y with unknown joint distribution P (x, y).…”

Section: Statistical Learning In a Nutshellmentioning

confidence: 99%

“…consisting of l ∈ N independent and identically distributed pairs (x i , y i ) ∈ X × Y sampled according to the distribution P (x, y) [19,100,135]. To measure the quality of a given prediction function f ∈ H, one can resort to the expected, empirical, and the regularized risk, which we will describe next.…”

Section: Statistical Learning In a Nutshellmentioning

confidence: 99%

From Supervised to Unsupervised Support Vector Machines and Applications in Astronomy

Gieseke

2013

Künstl Intell

View full text Add to dashboard Cite

A common task in the field of machine learning is the classification of objects. The basis for such a task is usually a training set consisting of patterns and associated class labels. A typical example is, for instance, the automatic classification of stars and galaxies in the field of astronomy. Here, the training set could consist of images and associated labels, which indicate whether a particular image shows a star or a galaxy. For such a learning scenario, one aims at generating models that can automatically classify new, unseen images. In the field of machine learning, various classification schemes have been proposed. One of the most popular ones is the concept of support vector machines, which often yields excellent classification results given sufficient labeled data.However, for a variety of real-world tasks, the acquisition of sufficient labeled data can be quite time-consuming. In contrast to labeled training data, unlabeled one can often be obtained easily in huge quantities. Semi-and unsupervised techniques aim at taking these unlabeled patterns into account to generate appropriate models. In the literature, various ways of extending support vector machines to these scenarios have been proposed. One of these ways leads to combinatorial optimization tasks that are difficult to address.In this thesis, several optimization strategies will be developed for these tasks that (1) aim at solving them exactly or (2) aim at obtaining (possibly suboptimal) candidate solutions in an efficient kind of way. More specifically, we will derive a polynomial-time approach that can compute exact solutions for special cases of both tasks. This approach is among the first ones that provide upper runtime bounds for the tasks at hand and, thus, yield theoretical insights into their computational complexity. In addition to this exact scheme, two heuristics tackling both problems will be provided. The first one is based on least-squares variants of the original tasks whereas the second one relies on differentiable surrogates for the corresponding objective functions. While direct implementations of both heuristics are still computationally expensive, we will show how to make use of matrix operations to speed up their execution. This will result in two optimization schemes that exhibit an excellent classification and runtime performance.Despite these theoretical derivations, we will also depict possible application domains of machine learning methods in astronomy. Here, the massive amount of data given for today's and future projects renders a manual analysis impossible and necessitates the use of sophisticated techniques. In this context, we will derive an efficient way to preprocess spectroscopic data, which is based on an adaptation of support vector machines, and the benefits of semi-supervised learning schemes for appropriate learning tasks will be sketched. As a further contribution to this field, we will propose the use of so-called resilient algorithms for the automatic data analysis taking place aboard toda...

show abstract

Introduction to Statistical Learning Theory

Cited by 346 publications

References 67 publications

Can under-exploited structure of original-classes help ECOC-based multi-class classification?

Can under-exploited structure of original-classes help ECOC-based multi-class classification?

A sequential sampling strategy for adaptive classification of computationally expensive data

From Supervised to Unsupervised Support Vector Machines and Applications in Astronomy

Contact Info

Product

Resources

About