Mining the Web for Collocations: IR Models of Term Associations

Verma, Rakesh M.; Vuppuluri, Vasanthi; Nguyen, An D.; Mukherjee, Arjun; Mammar, Ghita; Baki, Shahryar; Armstrong, Reed

doi:10.1007/978-3-319-75477-2_11

Cited by 3 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the output n-grams are given to the collocation and idiom detection algorithms. Collocation and idiom extraction has been done by the algorithm given by (Verma et al, 2016) 1 and (Verma and Vuppuluri, 2015). For part of speech tagging we combined NLTK's regex tagger with NLTK's N-Gram Tagger to have a better performance on POS tagging.…”

Section: Ice -Architecture and Algorithmsmentioning

confidence: 99%

“…We also compared our idiom extraction with AMALGr method on their dataset and the highest F1-score achieved by ICE was 95% compared to 67.42% for AMALGr. For detailed comparison of ICE's collocation and idiom extraction algorithm with existing tools, please refer to (Verma et al, 2016) and (Verma and Vuppuluri, 2015).…”

Section: Ice -Architecture and Algorithmsmentioning

confidence: 99%

“…Many scientists from diverse fields have worked on the challenging tasks of automated collocation and idiom extraction, e.g., see (Garg and Goyal, 2014;Seretan, 2013;Verma and Vuppuluri, 2015;Verma et al, 2016) and the references contained therein, yet there is no multi-purpose, ready-to-use, and flexible system for extracting these phrases. Collocation and its special forms, such as idioms, can be useful in many important tasks, e.g., summarization (Barrera and Verma, 2012), question-answering (Barrera et al, 2011), language translation, topic segmentation, authorial style, and so on.…”

Section: Introductionmentioning

confidence: 99%

“…For collocation extraction, ICE uses the IR models and techniques introduced by (Verma et al, 2016). These methods include: dictionary search, online dictionaries, a substitution method that compares the Bing hit counts of a phrase against the Bing hit counts of new phrases obtained by substituting the component words of the phrase one at a time to determine the "adherence factor" of the component words in a collocation, and two methods that try to measure the probability of association of the component words again using hit counts.…”

Section: Introductionmentioning

confidence: 99%

“…These methods include: dictionary search, online dictionaries, a substitution method that compares the Bing hit counts of a phrase against the Bing hit counts of new phrases obtained by substituting the component words of the phrase one at a time to determine the "adherence factor" of the component words in a collocation, and two methods that try to measure the probability of association of the component words again using hit counts. In (Verma et al, 2016), the authors created a gold-standard dataset of collocations by taking 100 sentences at random from the Wiki50 dataset and manually annotating them for collocations (including idioms) using eight volunteers, who used the Oxford Dictionary of Collocations and Oxford Dictionary of Idioms. Each sentence was given to two annotators, who were given 25 sentences each for annotation, and their work was checked and corrected afterwards by two other people.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics

2017

View full text Add to dashboard Cite

ii Introduction This volume contains the contributions accepted for Software Demonstrations at the EACL 2017 conference which takes place from 4th to 7th of April 2017 in Valencia, Spain.The Demonstrations program primary aim is to present the software systems and solutions related to all areas of computational linguistics and natural language processing. This program encourages the early exhibition of research prototypes, but also includes interesting mature systems.The number of submissions for software demonstrations exceeded, again, the number of available slots. This confronted us with the situation where we were forced to introduce a strict selection process. The selection process was completed following the predefined criteria of relevance for EACL 2017 conference: novelty, overall usability, and applicability to more than one linguistic level and to more than one computational linguistic field/task. We would like to thank the general chairs and the local organizers, without whom it would have been impossible to put together such a strong demonstrations program.We hope you will find this proceedings useful and that the software systems presented in this proceedings will inspire new ideas and approaches in your future work. AbstractIn semantic parsing, natural language questions map to meaning representation language (MRL) expressions over some fixed vocabulary of predicates. To do this reliably, one must guarantee that for a wide class of natural language questions (the so called semantically tractable questions), correct interpretations are always in the mapped set of possibilities. Here we demonstrate the system COVER which significantly clarifies, revises and extends the notion of semantic tractability. COVER is written in Python and uses NLTK.

show abstract

Section: Ice -Architecture and Algorithmsmentioning

confidence: 99%

Section: Ice -Architecture and Algorithmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics

2017

View full text Add to dashboard Cite

show abstract

A Lookup-Free Approach to Knowledge Extraction from News Feeds

Little¹,

Painter²

2016

RCS

View full text Add to dashboard Cite

Identifying topics without also introducing external assumptions is a major challenge for supervised learning techniques, which by definition classify texts according to precepts. Such approaches identify the presence of preclassified ideas, but cannot identify new ideas. In this paper, we present the results of applying a well understood unsupervised learning technique, in an innovative way, to news feeds analysis. We identify frequent sets of words using the A-Priori algorithm, and grade those sets according to the significance of the Association Rules that they imply. Such sets of words identify common themes in news feeds autonomously, with stopwords as the only added input. We present in detail this methodology and validate it by examining the identified ideas for their ability to identify actual news.

show abstract

ICE: Idiom and Collocation Extractor for Research and Education

Vuppuluri¹,

Baki²,

Nguyen³

et al. 2017

Proceedings of the Software Demonstrations of the 15th Conference Of the European Chapter of the Association for Comp

Self Cite

View full text Add to dashboard Cite

Collocation and idiom extraction are wellknown challenges with many potential applications in Natural Language Processing (NLP). Our experimental, open-source software system, called ICE, is a python package for flexibly extracting collocations and idioms, currently in English. It also has a competitive POS tagger that can be used alone or as part of collocation/idiom extraction. ICE is available free of cost for research and educational uses in two user-friendly formats. This paper gives an overview of ICE and its performance, and briefly describes the research underlying the extraction algorithms.

show abstract

Mining the Web for Collocations: IR Models of Term Associations

Cited by 3 publications

References 20 publications

Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics

Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics

A Lookup-Free Approach to Knowledge Extraction from News Feeds

ICE: Idiom and Collocation Extractor for Research and Education

Contact Info

Product

Resources

About