Text categorization for multiple users based on semantic features from a machine-readable dictionary

Abstract

The text categorization module described here provides a front-end filtering function for the larger DR-LINK text retrieval system [Liddy and Myaeing 1993]. The model evaluates a large incoming stream of documents to determine which documents are sufficiently similar to a profile at the broad subject level to warrant more refined representation and matching. To accomplish this task, each substantive word in a text is first categorized using a feature set based on the semantic Subject Field Codes (SFCs) assigned to individual word senses in a machine-readable dictionary. When tested on 50 user profiles and 550 megabytes of documents, results indicate that the feature set that is the basis of the text categorization module and the algorithm that establishes the boundary of categories of potentially relevant documents accomplish their tasks with a high level of performance. This means that the category of potentially relevant documents for most profiles would contain at least 80% of all documents later determined to be relevant to the profile. The number of documents in this set would be uniquely determined by the system's category-boundary predictor, and this set is likely to contain less than 5% of the incoming stream of documents.

Information Retrieval Systems: A Perspective on Human Computer Interaction

“…All these research endeavours have encouraged other researchers to continue and expand this work by synthesizing term level statistical techniques with epiphanic semantic processing in order to improve efficiency and effectiveness of automatic text categorization systems (Liddy, Paik & Yu, 1994). All these approaches are noteworthy efforts to address the differences between term expressions and term meanings however machine understanding and learning still rudimentarily remains an approximation of the anthropologic ability to read and understand.…”

A multilevel approach to intelligent information filtering

“…Standard statistical approaches exist for automated thesaurus generation [Salton and McGill 1983]. Several other approaches based on machine learning and NLP techniques have also been reported in the literature for automatic term discovery [Futrelle et al 1994;Guntzer et al 1988] and refinement [Liddy et al 1994]. Of course, users should also have the option to introduce new terms to suit their individual needs.…”

Use of a visual word dictionary for topic discovery in images

“…Many text classification problems [4], [5], [6] run the learning algorithm on words from a standard dictionary. The use of a dictionary allows the use of only standard words and thus reduces unwanted noise that can come in the forms described above.…”