Erd'14

Abstract

In this paper we overview the 2014 Entity Recognition and Disambiguation Challenge (ERD'14), which took place from March to June 2014 and was summarized in a dedicated workshop at SIGIR 2014. The main goal of the ERD challenge was to promote research in recognition and disambiguation of entities in unstructured text. Unlike many past entity linking challenges, no mention segmentations were given to the participating systems for a given document. Participants were asked to implement a web service for their system to minimize human involvement during evaluation and to enable measuring the processing times. The challenge has attracted a lot of interest (over 100 teams registered, and 27 of those submitted final results).In this paper we cover the task definition, issues encountered during annotation, and provide a detailed analysis of all the participating systems. Specifically, we show how we adapted the pooling technique to address the difficulties of gathering annotations for the entity linking task. We also summarize the ERD workshop that followed the challenge, including the oral and poster presentations as well as the invited talks.

Lessons learnt from the Named Entity rEcognition and Linking (NEEL) challenge series

“…The Entity Recognition and Disambiguation (ERD) challenge [17] was a benchmark initiative organised in 2015 as part of the SIGIR conference 9 with the focus of enabling entity-based information retrieval. For such a task, a system needs to be able to index documents based on entities, rather than words, and to identify which entities satisfy a given query.…”

“…In this phase manual annotations were performed using an annotation tool (e.g. CrowdFlower for the 2014 challenge dataset, 16 and GATE [25] for the 2015 challenge dataset 17 ). The annotators were asked to analyse the annotations generated in Phase 1 by adding or removing entity annotations as required.…”

“…Numerous initiatives have contributed to the progress in the field broadly covering different types of textual content (and thus going beyond the boundaries of tweets). For example TAC-KBP [55] has established a yearly challenge in the field covering newswire, websites, discussion forum posts, ERD [17] with search queries content, and SemEval [58] with technical manuals and reports.…”

Automatic Gloss Finding for a Knowledge Base using Ontological Constraints

“…In information retrieval, several recent approaches to query expansion make use of glosses [10,13] to improve the performance of ad-hoc information retrieval, based on a semantic, entity-rich representation of queries and documents. More generally, in the entity linking (EL) task [21,8], named entity mentions in text are mapped onto canonicalized nodes in the KB, thus disambiguating the named entity mentions. Classical approaches to EL make heavy Figure 1: Graph constructed by GLOFIN for the gloss finding problem using lexical matches and ontological constraints (e.g., mutual exclusion, subsumption etc).…”

Semantic similarity analysis and application in knowledge graphs

“…book or movie), recognizing the entity class can help to determine the corresponding entity in KGs. Therefore, recent researches (Guo et al, 2013;Sil and Yates, 2013) and challenges (Carmel et al, 2014b) are proposed to perform NER and NEL jointly. Designing models to represent the relations between entity context (surrounding words) and entity types is one of the main ideas to study NER and NEL jointly.…”