Networks and Natural Language Processing

Abstract

edge types have been used. Text units of various sizes and characteristics can be added as vertices in the graph, for example, words, collocations, word senses, entire sentences, or even entire documents. Note that the graph nodes do not have to belong to the same category. For example, both sentences and words can be added as vertices in the same graph. Edges can represent cooccurrence (such as two words that appear in the same sentence or in the same dictionary definition), collocation (for example, two words that appear immediately next to each other or that may be separated by a conjunction), syntactic structure (for example, the parent and child in a syntactic dependency), and lexical similarity (for example, cosine between the vector representations of two sentences).In terms of graph-based algorithms, the main methods used so far can be classified into: (1) semisupervised classification (Zhu and Ghahramani 2002;Zhu and Lafferty 2005;Toutanova, Manning, and Ng 2004;Radev 2004;Otterbacher, Erkan, and Radev 2005), where random walks or relaxation are applied on mixed sets of labeled and unlabeled nodes; (2) network analysis (Masucci and Rodgers 2006;, Caldeira et al. 2006), where network properties such as diameter, centrality, and so on, are calculated; (3) graph-based clustering methods (Pang and Lee 2004, Widdows andDorow 2002), such as min-cut methods; (4) minimum spanning-tree algorithms (McDonald et al. 2005).In this article, we overview several I n a cohesive text, language unitswhether they are words, phrases, or entire sentences-are connected through a variety of relations, which contribute to the overall meaning of the text and maintain the cohesive structure of the text and the discourse unity. Since the early ages of artificial intelligence, associative or semantic networks have been proposed as representations that enable the storage of such language units and the relations that interconnect them and that allow for a variety of inference and reasoning processes, simulating some of the functionalities of the human mind. The symbolic structures that emerge from these representations correspond naturally to graphs-where text constituents are represented as vertices and their interconnecting relations form the edges in the graph.The last decade has brought on a number of exciting research papers that apply graph-based methods to an increasingly large range of natural language problems, ranging from lexical acquisition to text summarization. In this article, we overview several of these methods and their application to natural language processing. To reflect the fact that the algorithms and representations originate in different communities-natural language processing and graph theory-we will be using a dual vocabulary to describe these methods: networks are graphs, nodes are vertices, and links are edges.In terms of graph-based representations, depending on the natural language-processing application, a variety of node and sentence John likes green apples, the main predicate is likes, which takes tw...

Similarity computation using semantic networks created from web-harvested data

“…In Radev and Mihalcea (2008) and Mihalcea and Radev (2011), an overview of network-based approaches is presented for a number of NLP problems. In Radev and Mihalcea (2008) and Mihalcea and Radev (2011), an overview of network-based approaches is presented for a number of NLP problems.…”

Natural Language Agreement in the Generation Mechanism based on Stratified Graphs

“…Graph theory is a well-studied sub-discipline of mathematics, with a large body of results and a large number of efficient algorithms that operate on graphs [20,9]. Despite the various existing linguistic theories, which lead to different ways of viewing sentence structure and therefore syntactic analysis, most linguists agree today that at the heart of sentence structure are the relations among words [18].…”

Network analysis of a corpus of undeciphered Indus civilization inscriptions indicates syntactic organization

“…Using the graph-theoretic paradigm to study a complex system has often revealed hitherto unsuspected patterns in it. While graph-based representation of texts has been used for some time in natural language processing tasks, such as, text parsing, disambiguation and clustering [3], the approach based on the new science of complex networks often asks questions from a different perspective that can shed new light on the organization of linguistic structure. For example, networks constructed on the basis of co-occurrence of words in sentences have been seen to exhibit (a) the small-world effect, i.e., a small average distance between any pair of arbitrarily chosen words, and (b) a scale-free distribution of the number of words a given word is connected to (i.e., its degree) [4].…”