Creating Welsh Language Word Embeddings

Abstract

Word embeddings are representations of words in a vector space that models semantic relationships between words by means of distance and direction. In this study, we adapted two existing methods, word2vec and fastText, to automatically learn Welsh word embeddings taking into account syntactic and morphological idiosyncrasies of this language. These methods exploit the principles of distributional semantics and, therefore, require a large corpus to be trained on. However, Welsh is a minoritised language, hence significantly less Welsh language data are publicly available in comparison to English. Consequently, assembling a sufficiently large text corpus is not a straightforward endeavour. Nonetheless, we compiled a corpus of 92,963,671 words from 11 sources, which represents the largest corpus of Welsh. The relative complexity of Welsh punctuation made the tokenisation of this corpus relatively challenging as punctuation could not be used for boundary detection. We considered several tokenisation methods including one designed specifically for Welsh. To account for rich inflection, we used a method for learning word embeddings that is based on subwords and, therefore, can more effectively relate different surface forms during the training phase. We conducted both qualitative and quantitative evaluation of the resulting word embeddings, which outperformed previously described word embeddings in Welsh as part of larger study including 157 languages. Our study was the first to focus specifically on Welsh word embeddings.

Usefulness of Digital Language Resources in Improving Native Language among Adults

“…Existing research has focused primarily on using digital language resources for second language and foreign learning among students and children [1,28,29]. There are very few studies that focused on native and particularly minority languages, their e-learning and analysis, such as, for example, the Welsh language [34]. The characteristics of native language usage compared to second language usage were only examined within the context of Slovene speakers' preferences for communication in online activities during second language learning [28].…”

Unsupervised Machine Learning to Identify Positive and Negative Themes in Jordanian mHealth Apps

“…Each item of the set is modelled as a finite mixture over an underlying set of topic probabilities (Sathi & Ramanujapura, 2016). To derive an accurate categorization of user feedback, the authors used the document-embedding widget before topic modelling to observe the embedding for each n-gram while employing the pre-trained fastText models for English and obtaining one vector per document (Alghamdi & Alfalqi, 2015;Corcoran et al, 2021). The LDAvis and multidimensional scaling (MDS) tools were used to evaluate the topics that emerged from the LDA, the LDAvis, and MDS.…”

Floating-Point Embedding: Enhancing the Mathematical Comprehension of Large Language Models

“…In tasks like predicting lexical complexity, leveraging Transformer models in conjunction with various traditional linguistic features has proven effective in enhancing the performance of deep learning systems [10]. Similarly, accommodating syntactic and morphological peculiarities is crucial, especially for languages like Welsh, a minority language, which necessitates adaptations to existing word embedding methods for optimal results [11]. Hence, the imperative lies in crafting specific word embedding methodologies tailored to the nuances of particular texts or tasks, recognizing the intricate interplay between linguistic structure and neural representations.…”