Inductive learning algorithms and representations for text categorization

Abstract

Text categorization -the assignment of natural language texts to one or more predefined categories based on their content -is an important component in many information organization and management tasks. We compare the effectiveness of five different automatic learning algorithms for text categorization in terms of learning speed, realtime classification speed, and classification accuracy. We also examine training set size, and alternative document representations. Very accurate text classifiers can be learned automatically from training examples. Linear Support Vector Machines (SVMs) are particularly promising because they are very accurate, quick to train, and quick to evaluate.

Text and Hypertext Categorization

“…A wide variety of learning approaches have been applied to TC, to name a few, Bayesian classification (Lewis and Ringuette 1994;Domingo and Pazzani 1996;Larkey and Croft 1996;Koller and Sahami 1997;Lewis 1998), decision trees (Weiss, Apte et al ;Fuhr and Buckley 1991;Cohen and Hirsh 1998;Li and Jain 1998), decision rule classifiers such as CHARADE (Moulinier and Ganascia 1996), or DL-ESC (Li and Yamanishi 1999), or RIPPER (Cohen and Hirsh 1998), or SCAR (Moulinier, Raskinis et al 1996), or SCAP-1 (Apté, Damerau et al 1994), multi-linear regression models (Yang and Chute 1994;Yang and Liu 1999), Rocchio method (Hull 1994;Ittner, Lewis et al 1995;Sable and Hatzivassiloglou 2000), Neural Networks (Schütze, Hull et al 1995;Wiener, Pedersen et al 1995;Dagan, Karov et al 1997;Ng, Goh et al 1997;Lam and Lee 1999;Ruiz and Srinivasan 1999), example based classifiers (Creecy 1991;Masand, Linoff et al 1992;Larkey 1999), support vector machines (Joachims 1998), Bayesian inference networks (Tzeras and Hartmann 1993;Wai and Fan 1997;Dumais, Platt et al 1998), genetic algorithms (Masand 1994;Clack, Farringdon et al 1997), and maximum entropy modelling (Manning and Schütze 1999).…”

“…Because this process is highly domain dependent and considering all possible combinations of tokens is impossible, many algorithms exist to define phrasal indexes. Although some researchers have reported an improvement in classification accuracy when using such indexes (depending on the quality of the generated phrases), a number of experimental results Apté, Damerau et al 1994;Dumais, Platt et al 1998) have not been uniformly encouraging, irrespective of whether the notion of "phrase" is motivated (i) syntactically, i.e. the phrase is such according to the grammar of the language ; or (ii) statistically, i.e.…”

“…Among these are the DIA association factor (Fuhr and Buckley 1991), chi-square (Yang and Pedersen 1997;Sebastiani, Sperduti et al 2000;Caropreso, Matwin et al 2001), NGL coefficient (Ng, Goh et al 1997;Ruiz and Srinivasan 1999), information gain Lewis and Ringuette 1994;Moulinier, Raskinis et al 1996;Yang and Pedersen 1997;Larkey 1998;Mladenic and Grobelnik 1998;Caropreso, Matwin et al 2001), mutual information (Larkey and Croft 1996;Wai and Fan 1997;Dumais, Platt et al 1998;Taira and Haruno 1999) odds ratio (Mladenic and Grobelnik 1998;Ruiz and Srinivasan 1999;Caropreso, Matwin et al 2001), relevancy score (Wiener, Pedersen et al 1995) and GSS coefficient (Galavotti, Sebastiani et al 2000). Three of the most popular methods are descrivbed briefly below.…”

Multi-topic Aspects in Clinical Text Classification

“…In the late '90s, Machine Learning techniques were successfully applied to Text Classification. Support Vector Machines were applied to Text Classification in [6,4]. Maximum Entropy Models were also applied in [8].…”

Multinomial Event Model Based Abstraction for Sequence and Text Classification

“…Following the previous works [14,15,10], we build binary classifiers for top ten most populous categories. In our experiment, stop words were not eliminated, and title words were not distinguished with body words.…”