Natural language processing in support of decision-making: phrases and part-of-speech tagging

Abstract

The use of natural language information can improve decision-making. Darwinian considerations suggest that language may have developed because it leads to improved decision making and survival, justifying the study of language's contribution to decision making. The study of information-based decision making within the context of evolution provides a view of information use that allows us to both describe the phenomenon of information use as well as to explain why an information use occurs as it does. Increasing information retrieval performance using phrases and part-of-speech (POS) information is one example of a type of decision-making performance that is improved when using this linguistic information. By studying a set of phrases used in a text retrieval system, we are able to show the relative effectiveness of using multi-term phrases as opposed to individual terms, as well as the relative worth of POS tagged terms or phrases, as opposed to untagged terms or phrases. An explanation is suggested for why POS tags contribute less to higher order grammatical constructs. We propose a measure of those needs for POS disambiguation that can be addressed by tagging; some example terms are analyzed using this measure, and specific degrees of ambiguity are proposed.

A Part-Of-Speech term weighting scheme for biomedical information retrieval

“…Chowdhury and McCabe utilized only POS to build nouns tagger [21] index and found that the overall identified by performance decreased less than 1% but the index size was tremendously reduced. Losee [22] found that nouns or noun phrases are more important for search engine “spies” than other POS categories. In the practical IR systems, stop words are generally removed before indexing due to the fact that most stop words are prepositions containing little information.…”

Is 1 noun worth 2 adjectives? Measuring relative feature utility

“…Part-of-speech tags, attached to a specific term, provides valuable information that can help disambiguate terms, allowing a system to determine which meaning of a term is represented by the term's presence (E. A. Fox, Nutter, Ahlswede, Evens, & Markowitz, 1988;Justeson & Katz, 1995;Losee, 2001;Rittman et al, 2004;Wilks & Stevenson, 1998). When the term bank occurs in a document and a query, having the term labeled as a noun in the query and meaning a river bank and labeled as a verb in the document meaning to bank a plane, the POS tagging allows the non-match to exclude documents with the term whose sense doesn't match the query sense.…”

“…What is the relative degree to which these linguistic and non-linguistic characteristics represent topicality and predict the utility of a document to a system user? Term roots may carry one or more meanings or topics, and the addition of contextual or supporting information, such as suffixes, part-of-speech tags, and larger contexts can contribute to the topicality, therefore improving document ordering (Bossong, 1989;Clement & Sharp, 2003;Losee, 2001) How does one measure how many of one feature or type of feature is equivalent in ordering power to another chosen feature or type of feature? The Relative Feature Utility may be used to empirically analyze the ordering effects of term stemming, the length of natural language phrases, the effect of using different part-of-speech labels, and various information retrieval or filtering assumptions.…”

The Semanference System: Better Search Results through Better Queries

“…nature" of queries devised using current methods (775), as well as the need for greater input and specificity of noun-based tags. However, he finds POS tagging of long phrases ineffective because of a corresponding loss of linguistic nuance (Losee, 2001). Losee's finding underscores the need for additional structural techniques to be applied to Web queries.…”