Extended Overview of CLEF HIPE 2020: Named Entity Processing on Historical Newspapers

Ehrmann, Maud; Romanello, Matteo; Flückiger, Alex; Clematide, Simon

doi:10.5281/zenodo.4117566

Cited by 6 publications

(9 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They found remarkable performance differences between noisy and non-noisy mentions, and that already as little noise as 0.1 severely hurts systems' abilities to predict an entity and may halve their performances. To sum up, whether focused on a single OCR version of text(s) [195], on different artificially-generated ones [79], or on the noise present in entities themselves [53], these studies clearly demonstrate how challenging OCR noise is for NER systems.…”

Section: Character Recognitionmentioning

confidence: 93%

“…Focusing specifically on entity processing, Hamdi et al [79,80] confronted a BiLSTM-based NER model with OCR outputs of the same text but of different qualities and observed a 30 percentage point loss in F-score when the character error rate increased from 7% to 20%. Finally, in order to assess the impact of noisy entities on NER during the CLEF-HIPE-2020 NE evaluation campaign on historical newspapers (HIPE-2020 for short), 7 Ehrmann et al [53] evaluated systems' performances on various entity noise levels, defined as the length-normalised Levenshtein distance between the OCR surface form of an entity and its manual transcription. They found remarkable performance differences between noisy and non-noisy mentions, and that already as little noise as 0.1 severely hurts systems' abilities to predict an entity and may halve their performances.…”

Section: Character Recognitionmentioning

confidence: 99%

“…Worth of mention is the evaluation of several NER systems on historical newspaper corpora spanning ca. 200 years, first with the study of Ehrmann et al [50], second on the occasion of the HIPE-2020 shared task [53]. Testing the hypothesis of the older the document, the lower the performance, both studies reveal a contrasted picture with non-linear F-score variations over time.…”

Section: Entity and Context Driftsmentioning

confidence: 99%

“…This paucity also affects systems' evaluation and comparison which, besides the lack of gold standards, is also characterised by fragmented and non-standardised evaluation approaches. The recently organised CLEF-HIPE-2020 shared task on NE processing in multilingual and historical newspapers is a first step towards alleviating this situation [53].…”

Section: Lack Of Resourcesmentioning

confidence: 99%

“…In the context of the CLEF-HIPE-2020 shared task [53], Dekhili et al [41] proposed different variations of a BiLSTM-CRF network, with and without the in-domain HIPE flair embeddings and/or an attention layer. The gains of adding one or the other or both are not easy to interpret, with uneven performances of the model variants across NE types.…”

Section: Input Representationmentioning

confidence: 99%

See 4 more Smart Citations

Named Entity Recognition and Classification on Historical Documents: A Survey

Ehrmann¹,

Hamdi²,

Pontes³

et al. 2021

Preprint

View full text Add to dashboard Cite

After decades of massive digitisation, an unprecedented amount of historical documents is available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve and explore information from this 'big data of the past'. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.

show abstract

Section: Character Recognitionmentioning

confidence: 93%

Section: Character Recognitionmentioning

confidence: 99%

Section: Entity and Context Driftsmentioning

confidence: 99%

Section: Lack Of Resourcesmentioning

confidence: 99%

Section: Input Representationmentioning

confidence: 99%

See 3 more Smart Citations

Named Entity Recognition and Classification on Historical Documents: A Survey

Ehrmann¹,

Hamdi²,

Pontes³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Introducing the HIPE 2022 Shared Task: Named Entity Recognition and Linking in Multilingual Historical Documents

Ehrmann

Romanello

Doucet

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We present the HIPE-2022 shared task on named entity processing in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 evaluation lab, this edition confronts systems with the challenges of dealing with more languages, learning domain-specific entities, and adapting to diverse annotation tag sets. HIPE-2022 is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop appropriate technologies to efficiently retrieve and explore information from historical texts. On such material, however, named entity processing techniques face the challenges of domain heterogeneity, input noisiness, dynamics of language, and lack of resources. In this context, the main objective of the evaluation lab is to gain new insights into the transferability of named entity processing approaches across languages, time periods, document types, and annotation tag sets.

show abstract