XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment

Abstract

Cross-lingual named-entity lexica are an important resource to multilingual NLP tasks such as machine translation and cross-lingual wikification. While knowledge bases contain a large number of entities in high-resource languages such as English and French, corresponding entities for lower-resource languages are often missing. To address this, we propose Lexical-Semantic-Phonetic Align (LSP-Align), a technique to automatically mine cross-lingual entity lexica from mined web data. We demonstrate LSP-Align outperforms baselines at extracting cross-lingual entity pairs and mine 164 million entity pairs from 120 different languages aligned with English. We release these cross-lingual entity pairs along with the massively multilingual tagged named entity corpus as a resource to the NLP community.http://data.statmt.org/xlent/ https://opus.nlpl.eu/

Language Varieties of Italy: Technology Challenges and Opportunities

“…15 For each corpus, a native speaker of each included language variety was asked to label a 14 https://iso639-3.sil.org/request/2008 -040. 15 We do not include XLEnt (El-Kishky et al, 2021) since it comprises cross-lingual named entities rather than texts. random sample of 50 texts (or parallel texts, in CCAligned and WikiMatrix) according to the labeling scheme and guidelines presented in Kreutzer et al (2022).…”