Improving mention detection for Basque based on a deep error analysis

Soraluze, Ander; Arregi, Olatz; Arregi, Xabier; Ilarraza, Arantza Díaz de

doi:10.1017/s1351324916000206

Cited by 6 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mentions and coreference chains have been annotated semi-automatically based on previous layers in a 46,383-word subset of the EPEC corpus. Automatically annotated mentions obtained by our mention detector [36] were first corrected by linguists; then, coreferent mentions were linked in clusters. It is freely available and can be downloaded at: ixa2.si.ehu.eus/epec-koref/epec-koref_v1.0.tgz.…”

Section: Mention Structures In Basquementioning

confidence: 99%

See 1 more Smart Citation

EUSKOR: End-to-end coreference resolution system for Basque

Soraluze

Arregi

et al. 2019

PLoS ONE

Self Cite

View full text Add to dashboard Cite

This paper describes the process of adapting the Stanford Coreference resolution module to the Basque language, taking into account the characteristics of the language. The module has been integrated in a linguistic analysis pipeline obtaining an end-to-end coreference resolution system for the Basque language. The adaptation process explained can benefit and facilitate other languages with similar characteristics in the implementation of their coreference resolution systems. During the experimentation phase, we have demonstrated that language-specific features have a noteworthy effect on coreference resolution, obtaining a gain in CoNLL score of 7.07 with respect to the baseline system. We have also analysed the effect that preprocessing has in coreference resolution, comparing the results obtained with automatic mentions versus gold mentions. When gold mentions are provided, the results increase 11.5 points in CoNLL score in comparison with results obtained when automatic mentions are used. The contribution of each sieve is analysed concluding that morphology is essential for agglutinative languages to obtain good performance in coreference resolution. Finally, an error analysis of the coreference resolution system is presented which have revealed our system’s weak points and help to determine the improvements of the system. As a result of the error analysis, we have enriched the Basque coreference resolution adding new two sieves, obtaining an improvement of 0.24 points in CoNLL F1 when automatic mentions are used and of 0.39 points when the gold mentions are provided.

show abstract

Section: Mention Structures In Basquementioning

confidence: 99%

“…We created a mention detection system [36], defining a set of hand-crafted rules that have been compiled into Finite State Transducers (FST). These FSTs are able to detect complex structures that should be identified as mentions.…”

Section: System Architecturementioning

confidence: 99%

EUSKOR: End-to-end coreference resolution system for Basque

Soraluze

Arregi

et al. 2019

PLoS ONE

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nevertheless, a rule-based coreference resolution system (Soraluze et al, 2015) and a machine learning based system (Soraluze et al, 2016) have been developed. Both of which used a rule-based mention detector (Soraluze et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference

2019

View full text Add to dashboard Cite

In many NLP applications like search and information extraction for named entities, it is necessary to find all the mentions of a named entity, some of which appear as pronouns (she, his, etc.) or nominals (the professor, the German chancellor, etc.). It is therefore important that coreference resolution systems are able to link these different types of mentions to the correct entity name. We evaluate state-of-theart coreference resolution systems for the task of resolving all mentions to named entities. Our analysis reveals that standard coreference metrics do not reflect adequately the requirements in this task: they do not penalize systems for not identifying any mentions by name to an entity and they reward systems even if systems find correctly mentions to the same entity but fail to link these to a proper name (she-the student-no name). We introduce new metrics for evaluating named entity coreference that address these discrepancies and show that for the comparisons of competitive systems, standard coreference evaluations could give misleading results for this task. We are, however, able to confirm that the state-of-the art system according to traditional evaluations also performs vastly better than other systems on the named entity coreference task.

show abstract

“…The part of the corpus we have used has about 45,000 words and it has been manually tagged at coreference level by two linguists (Ceberio et al, 2016). First of all, automatically tagged mentions obtained by a mention detector (Soraluze et al, 2016) have been corrected; then, coreferent mentions have been linked in clusters.…”

Section: Error Analysismentioning

confidence: 99%