Razpoznavanje imenskih entitet v slovenskem besedilu

Abstract

Članek predstavlja algoritem in implementacijo programa za razpoznavanje imen v slovenskem jeziku s pomočjo strojnega učenja. Nadzorovani pristop na osnovi pogojnih naključnih polj je naučen na označenem korpusu ssj500k. V korpusu, ki je prosto dostopen pod licenco Creative Commons CC-BY-NC-SA, so pri besednih pojavnicah poleg oblikoskladenjskih oznak in lem označena tudi imena organizacij, osebna, zemljepisna ter stvarna imena. Članek predstavlja vpliv na natančnost razpoznavanja ob uporabi oblikoskladenjskih oznak, leksikonov in konjunkcij sosednjih lastnosti. Ena od ugotovitev raziskave je, da so oblikoskladenjske oznake pri razpoznavanju entitet koristne. V kombinaciji z vsemi ostalimi lastnostmi doseže sistem na testni množici 74% natančnost in 72% priklic, pri čemer so najbolje razpoznana osebna imena, sledijo jim zemljepisna ter organizacijska in nazadnje stvarna imena. Novo spoznanje članka je tudi to, da lahko z delitvijo razreda vseh stvarnih imen na organizacije in preostala stvarna imena dosežemo boljše rezultate prepoznavanja tudi pri drugih razredih. Preizkusi na neodvisno označenih korpusi kažejo dobro posplošenost modela za osebna in zemljepisna imena. Programska oprema, narejena v raziskavi, je prosto dostopna pod licenco Apache 2.0 na naslovu http://ailab.ijs.si/~tadej/slner.zip, razvojne različice pa so na voljo na naslovuhttps://github.com/tadejs/slner.

The Second Cross-Lingual Challenge on Recognition, Normalization, Classification, and Linking of Named Entities across Slavic Languages

“…In 2010, the NEWS Workshop included a shared task on Transliteration Mining (Kumaran et al, 2010), i.e., mining of names from parallel corpora, in English, Chinese, Tamil, Russian, and Arabic. Research on NER focusing on Slavic languages includes NER for Croatian (Karan et al, 2013;Ljubešić et al, 2013); NER in Croatian tweets (Baksa et al, 2017); a manually annotated NE corpus for Croatian (Agić and Ljubešić, 2014); NER in Slovene (Štajner et al, 2013;Ljubešić et al, 2013); a Czech corpus of 11K annotated NEs (Ševčíková et al, 2007); NER for Czech (Konkol and Konopík, 2013); tools and resources for fine-grained annotation of NEs in the National Corpus of Polish (Waszczuk et al, 2010;Savary and Piskorski, 2011); lemmatization of NEs for Polish (Piskorski et al, 2009;Marcińczuk, 2017). Shared tasks on NER for Polish were organized under the umbrella of POLEVAL 2 Kobyliński, 2018, 2020) and LESZCZE 3 campaigns.…”

Slav-NER: the 4th Cross-lingual Challenge on Recognition, Normalization, Classification, and Linking of Named Entities across Slavic languages

“…Research on NE focusing on Slavic languages includes NE recognition for Croatian (Karan et al, 2013;Ljubešić et al, 2013), NE recognition in Croatian tweets (Baksa et al, 2017), a manually annotated NE corpus for Croatian (Agić and Ljubešić, 2014), NE recognition in Slovene (Štajner et al, 2013;Ljubešić et al, 2013), a Czech corpus of 11K annotated NEs (Ševčíková et al, 2007), NER for Czech (Konkol and Konopík, 2013), tools and resources for fine-grained annotation of NEs in the National Corpus of Polish (Waszczuk et al, 2010;Savary and Piskorski, 2011), NER shared tasks for Polish organized under the umbrella of POLEVAL 2 (Ogrodniczuk andŁukasz Kobyliński, 2018, 2020) and LESZCZE 3 campaigns, recent shared tasks on NE Recognition in Russian (Starostin et al, 2016;Artemova et al, 2022), the latter utilizing the NEREL dataset (a Russian dataset for named entity recognition and relation extraction, described in Loukachevitch et al, 2021)…”

Levels of Annotation in the Slovene Training Corpus ssj500k 2.2

“…Approximately a third of ssj500k 2.2 (9,478 sentences) was manually annotated with named entity annotations in the WebAnno tool, with the aim of developing a named entity extractor for the Slovene language based on machine learning [20]. The annotation distinguished five types of NE: Person (per), Person Derivative (deriv-per), Location (loc), Organization (org), and Miscellaneous (misc).…”