Evaluation of WSD Systems

Palmer, Martha; Ng, Hwee Tou; Dang, Hoa Trang

doi:10.1007/978-1-4020-4809-8_4

Cited by 16 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scores were calculated using the standard coverage, precision, recall, and F-score measures as used in word sense disambiguation (Palmer et al, 2007):…”

Section: Task Definitionmentioning

confidence: 99%

SemEval-2017 Task 7: Detection and Interpretation of English Puns

Miller¹,

Hempelmann²,

Gurevych³

2017

Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)

View full text Add to dashboard Cite

A pun is a form of wordplay in which a word suggests two or more meanings by exploiting polysemy, homonymy, or phonological similarity to another word, for an intended humorous or rhetorical effect. Though a recurrent and expected feature in many discourse types, puns stymie traditional approaches to computational lexical semantics because they violate their one-sense-percontext assumption. This paper describes the first competitive evaluation for the automatic detection, location, and interpretation of puns. We describe the motivation for these tasks, the evaluation methods, and the manually annotated data set. Finally, we present an overview and discussion of the participating systems' methodologies, resources, and results.

show abstract

“…Scores were calculated using the standard coverage, precision, recall, and F-score measures as used in word sense disambiguation (Palmer et al, 2007):…”

Section: Task Definitionmentioning

confidence: 99%

SemEval-2017 Task 7: Detection and Interpretation of English Puns

Miller¹,

Hempelmann²,

Gurevych³

2017

Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)

View full text Add to dashboard Cite

show abstract

“…IAA in Senseval-2 Verb Lexical Sample by expert annotators (Palmer et al, 2006) is 71% and in Senseval-3 by Contributors over the Web (Mihalcea et al, 2004) IAA is 67.3% with a Macro Average-K of 0.35. However in these tasks the IAA was computed somewhat differently (Palmer et al, 2006…”

Section: Conclusion and Further Workmentioning

confidence: 99%

SentIta and Doxa: Italian Databases and Tools for Sentiment Analysis Purposes

Pelosi¹

2015

Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

View full text Add to dashboard Cite

show abstract

“…Unfortunately, it does not really give a large improvement in performance as compared to the strategy of choosing most frequent senses, see Palmer et al (2006) and Sec. 8.2 for more details.…”

Section: Axioms Derived From Wordnetmentioning

confidence: 99%

Integration of World Knowledge for Natural Language Understanding

Ovchinnikova¹

2012

Atlantis Thinking Machines

View full text Add to dashboard Cite

Aims and scope of the seriesThis series publishes books resulting from theoretical research on and reproductions of general Artificial Intelligence (AI). The book series focuses on the establishment of new theories and paradigms in AI. At the same time, the series aims at exploring multiple scientific angles and methodologies, including results from research in cognitive science, neuroscience, theoretical and experimental AI, biology and from innovative interdisciplinary methodologies.All books in this series are co-published with Springer.For more information on this series and our other book series, please visit our website at:www.atlantis-press.comForeword Inference-based natural language understanding (NLU) was a thriving area of research in the 1970s and 1980s. It resulted in good theoretical work and in interesting small-scale systems. But in the early 1990s it foundered on three difficulties:• Parsers were not accurate enough to produce predicate-argument relations reliably, so that inference had no place to start.• Inference processes were not efficient enough nor accurate enough.• There was no large knowledge base designed for NLU applications.The first of these difficulties has been overcome by progress in statistical parsing. The second problem is one that many people, including Ekaterina Ovchinnikova, are working on now. The research described in this volume addresses the third difficulty, and indeed shows considerable promise in overcoming it. For this reason, I believe Dr. Ovchinnikova's work has a real potential to reignite interest in inference-based NLU in the computational linguistics community.A key notion in her work is that there already exists sufficient world knowledge in a variety of resources, at a level of precision that enables their translation into formal logic. To my mind, the most important of these are WordNet and FrameNet, especially the latter, and she describes the kind of information one can get out of these resources. She exploits in particular the hierarchical information and the glosses in WordNet, generating 600,000 axioms.She also describes how one can utilize FrameNet to generate about 50,000 axioms representing relations between words and frames, and about 5000 axioms representing frameframe relations. Her analysis of FrameNet is quite thorough, and I found this part of her work inspiring.She also critically discusses foundational ontologies such as DOLCE, SUMO. and Open-Cyc, and domain-specific ontologies of the sort being constructed for the Semantic Web. v vi Integration of World Knowledge for Natural Language UnderstandingShe examines the problems raised by semi-formal ontologies, like YAGO and ConceptNet, which have been gleaned from text or Netizens and which may be more difficult to translate into formal logic. She also shows how to use distributional data for a default mode of processing when the required knowledge is not available.Her use of knowledge from a variety of resources, combined into a single system, leads to the very hard problem of ensuring consisten...

show abstract

Evaluation of WSD Systems

Cited by 16 publications

References 23 publications

SemEval-2017 Task 7: Detection and Interpretation of English Puns

SemEval-2017 Task 7: Detection and Interpretation of English Puns

SentIta and Doxa: Italian Databases and Tools for Sentiment Analysis Purposes

Integration of World Knowledge for Natural Language Understanding

Contact Info

Product

Resources

About