Robust Classifiers without Robust Features

Katz, Alan; Gately, Michael T.; Collins, D.R.

doi:10.1162/neco.1990.2.4.472

Cited by 14 publications

(14 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the feature-oriented taxonomic approaches to contextual computing in the machine learning domain, we highlight the works of Katz et al, Turney, or Widdows [10,16,19,18]. With our focus on STM exploitation, we differentiate between discourse-contextual features that already encode (spatio-)temporal inter-stroke measurements and stroke-centric features that only contain local intra-stroke characteristics measured on the vertex level.…”

Section: Unfolding the Scene Historymentioning

confidence: 99%

Sketch-Based Interfaces: Exploiting Spatio-temporal Context for Automatic Stroke Grouping

Dickmann¹,

Lensing²,

Porzel³

et al. 2010

Smart Graphics

View full text Add to dashboard Cite

Abstract. In this paper, we investigate how discourse context in the form of short-term memory can be exploited to automatically group consecutive strokes in digital freehand sketching. With this machine learning approach, no database of explicit object representations is used for template matching on a complete scene-instead, grouping decisions are based on limited spatio-temporal context. We employ two different classifier formalisms for this time series analysis task, namely Echo State Networks (ESNs) and Support Vector Machines (SVMs). ESNs present internal-state classifiers with inherent memory capabilities. For the conventional static SVM, short-term memory is supplied externally via fixed-length feature vector expansion. We compare the respective setup heuristics and conduct experiments with two exemplary problems. Promising results are achieved with both formalisms. Yet, our experiments indicate that using ESNs for variable-length memory tasks alleviates the risk of overfitting due to non-expressive features or improperly determined temporal embedding dimensions.

show abstract

Section: Unfolding the Scene Historymentioning

confidence: 99%

Sketch-Based Interfaces: Exploiting Spatio-temporal Context for Automatic Stroke Grouping

Dickmann¹,

Lensing²,

Porzel³

et al. 2010

Smart Graphics

View full text Add to dashboard Cite

show abstract

“…At classification time, a meta-classifier can then be used to switch between classifiers according to the current context (Sammut et al, 1992;Katz, Gately & Collins, 1990). The application of stable concepts to on-line classification used in this paper (Sections 3.2.2 and 6.2) uses a similar switching approach.…”

Section: Related Workmentioning

confidence: 99%

Untitled

Harries¹,

Sammut²,

Horn³

1998

Machine Learning

132

View full text Add to dashboard Cite

Abstract. Concept drift due to hidden changes in context complicates learning in many domains including financial prediction, medical diagnosis, and communication network performance. Existing machine learning approaches to this problem use an incremental learning, on-line paradigm. Batch, off-line learners tend to be ineffective in domains with hidden changes in context as they assume that the training set is homogeneous. An off-line, meta-learning approach for the identification of hidden context is presented. The new approach uses an existing batch learner and the process of contextual clustering to identify stable hidden contexts and the associated context specific, locally stable concepts. The approach is broadly applicable to the extraction of context reflected in time and spatial attributes. Several algorithms for the approach are presented and evaluated. A successful application of the approach to a complex flight simulator control task is also presented.

show abstract

“…Michalski (1987) was one of the first to formulate it; he suggested a specialized two-tiered representation formalism to represent different aspects of context-dependent concepts (see also Bergadano et al, 1992). Recently, context dependence has been recognized as a problem in a number of practical machine learning projects (e.g., Katz et al, 1990;Turney, 1993;Turney & Halasz, 1993;Watrous, 1993;Watrous & Towell, 1995; see also Kubat & Widmer, 1996). There, various techniques for context handling were developed.…”

Section: Motivationmentioning

confidence: 99%

“…A similar approach was taken to adapt a classifier to different speakers in speech recognition (Watrous, 1993). Earlier, Katz et al (1990) had described a two-stage neural network classifier, where a higher-level network learned to switch between a set of n base-level classifiers. The application domain was the recognition of targets on infrared and daytime television images.…”

Section: Related Workmentioning

confidence: 99%

Untitled

Widmer

1997

Machine Learning

View full text Add to dashboard Cite

Abstract. The article deals with the problem of learning incrementally ('on-line') in domains where the target concepts are context-dependent, so that changes in context can produce more or less radical changes in the associated concepts. In particular, we concentrate on a class of learning tasks where the domain provides explicit clues as to the current context (e.g., attributes with characteristic values). A general two-level learning model is presented that effectively adjusts to changing contexts by trying to detect (via 'meta-learning') contextual clues and using this information to focus the learning process. Context learning and detection occur during regular on-line learning, without separate training phases for context recognition. Two operational systems based on this model are presented that differ in the underlying learning algorithm and in the way they use contextual information: MetaL(B) combines meta-learning with a Bayesian classifier, while MetaL(IB) is based on an instance-based learning algorithm. Experiments with synthetic domains as well as a number of 'real-world' problems show that the algorithms are robust in a variety of dimensions, and that meta-learning can produce substantial increases in accuracy over simple object-level learning in situations with changing contexts.Keywords: Meta-learning, on-line learning, context dependence, concept drift, transfer MotivationThe fact that concepts in the real world are not eternally fixed entities or structures, but can have a different appearance or definition or meaning in different contexts has only gradually been recognized as a relevant problem in concept learning. Michalski (1987) was one of the first to formulate it; he suggested a specialized two-tiered representation formalism to represent different aspects of context-dependent concepts (see also Bergadano et al., 1992). Recently, context dependence has been recognized as a problem in a number of practical machine learning projects (e.g., Katz et al., 1990;Turney, 1993;Turney & Halasz, 1993;Watrous, 1993;Watrous & Towell, 1995; see also Kubat & Widmer, 1996). There, various techniques for context handling were developed. Most of these methods either assume that contextual attributes are explicitly identified by the user, or require separate pre-training phases on special data sets that are cleanly separated according to context.We are studying the effects of context dependence and changing contexts in the framework of incremental (or on-line) learning, and we are interested in learners that can adapt to different contexts without explicit help from a teacher. The scenario is as follows: assume that a learner is learning on-line from a stream of incoming (labeled) examples. Assume further that the concepts of interest depend on some (maybe hidden) context, and that changes in this context can induce corresponding changes in the target concepts. As a simple example, consider weather prediction rules, which may vary drastically with the change of seasons. The visible effects of such changes ...

show abstract

Robust Classifiers without Robust Features

Cited by 14 publications

References 3 publications

Sketch-Based Interfaces: Exploiting Spatio-temporal Context for Automatic Stroke Grouping

Sketch-Based Interfaces: Exploiting Spatio-temporal Context for Automatic Stroke Grouping

Untitled

Untitled

Contact Info

Product

Resources

About