Strategies for building wordnets for under-resourced languages: The case of African languages

Abstract

The African Wordnet Project (AWN) aims at building wordnets for five African languages: Setswana, isiXhosa, isiZulu, Sesotho sa Leboa (also referred to as Sepedi or Northern Sotho) and Tshivenda. Currently, the so-called expand model, based on the structure of the English Princeton WordNet (PWN), is used to continually develop the African Wordnets manually. This is a labour-intensive work that needs to be performed by linguistic experts, guided by several considerations such as the level of lexicalisation of a term in the African language. Up to now, linguists were responsible for identifying and translating appropriate synsets without much help from electronic resources because in the case of African languages even basic resources such as computer readable and electronic bilingual wordlists are usually not freely available. Methods to speed up the manual development of synsets and ease the workload of the human language experts were recently investigated. These centred around utilising the minimal amount of information available in bilingual dictionaries to identify synsets in the PWN that should be included in the AWN, transferring information from dictionaries to the wordnet and presenting the potential synsets to linguists for final approval and inclusion in the wordnets. In this article, we describe the methodology developed for building the African Wordnets, a potentially significant resource for natural language processing applications. Available resources that could be taken advantage of and resources that had to be developed are investigated, and initial results and future plans are explained.

Computational Analysis versus Human Intuition: A Critical Comparison of Vector Semantics with Manual Semantic Classification in the Context of Plains Cree

“…The use of such a simplification of WordNet's semantic relations significantly reduces the amount of time necessary to semantically classify each word, as only a direct correspondence to the relevant WordNet synset would be necessary for a lexical item in the target language to be considered classified, with first-pass hypernymy and hyponymy relationships constructed indirectly by populating synsets. Using this method, manual classification of dictionary items can provide a basic semantic ontology of the target language at a rate of 400-500 word types daily per annotator, compared with a rate of ~1000 synsets per year reported by Bosch and Griesel during their creation of full WordNets for low-resource South African Bantu languages (Bosch and Griesel, 2017). This skeletal form of WordNet also provides the benefit of requiring substantially less linguistic background knowledge to effectively use, reducing the need for lengthy annotator training sessions.…”

“…Although initially developed for English, the WordNet approach for semantic classification has since become a staple in modern lexicography, with WordNets of varying size and complexity existing for many prominent global and national majority languages, such as German with GermaNet (Hamp and Feldweg, 1997;Hinrich and Hinrichs, 2010), Finnish with FinnWordNet (Lindén and Niemi, 2014), and Korean with KorLex (Aesun Yoon et al, 2009), among dozens of others. However, while semantic classifications such as these have become relatively commonplace among prominent majority languages in the developed world, they remain a rarity among underdocumented or otherwise poorly resourced languages (Bosch and Griesel, 2017). Using existing, conventional lexical resources, we provide here a holistic comparison between a manual method in semantic classification using a WordNet-based ontology and an automatic computational method via vector semantics, with respect to the efficacy and precision of both methods.…”

Jumping Finite Automata for Tweet Comprehension

“…Expansion approach -existing synsets from a reference WN are used as a guide to create corresponding synsets in a new WN, by gathering applicable words that represent the meaning of the synset. This approach has been shown to be suitable for under resourced languages [13].…”

“…We highlight a number of WordNets developed based on expansion approach that are relevant to the development of a WordNet for the language of Kenya used in this work. The WordNets include: EuroWordNet developed by linking several European languages to English WordNet [14]; Persian WordNet [15]; Finnish WordNet [15]; Polish WordNet [16] and African WordNet (AWN) [13] created by aligning several languages spoken in Southern Africa. A number of tools were used in development of AWN including DEBVisDic 6 editor tools for linguists building AWN.…”

“…POP raw tweet token from jfa 10: if jfa is empty then 11: jfa recognises raw tweet 12: increment matched jfas by 1 13: return status…”