Deep Learning Transformer Architecture for Named Entity Recognition on Low Resourced Languages: State of the art results

Hanslo, Ridewaan

doi:10.15439/2022f53

Cited by 5 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These models were then evaluated against other neural networks and machine learning. Transformer architecture models' F1-score consistently surpassed neural networks' and machine learning's techniques [6], [7].…”

Section: Related Workmentioning

confidence: 99%

A Convolutional Neural Network Model for Setswana Named Entity Recognition

Chabalala,

Owolawi,

Ojo

2023

iCARTi

View full text Add to dashboard Cite

Named entity recognition (NER) is a key component of the core task of natural language processing (NLP). In order to represent language, neural networks have been used starting in the 2000s, which enhanced entity recognition outcomes. The Setswana language, in contrast, has never been used with neural networks, in particular convolutional neural networks (CNN). Recently, problems with NLP have been addressed using CNNs, and the results have been quite interesting. CNNs are frequently used in NLP due to their ease of training and reputation as the best in sequence labelling. They depict the interdependence of all conceivable word combinations. Given the difficulties in identifying named entities for South African languages, including Setswana, and the inadequacy of resources, this research proposes the use of CNN model to identify named entities for Setswana. The results obtained are benchmarked with traditional methods such as Conditional random fields (CRF). The performance metrics such as F1-Score are explored in establishing the magnitude of trust and reliability of the proposed model. The model is evaluated using data from the South African Centre for Digital Language Resources' Setswana NER dataset. Compared to the present CRF model, which had an F-score performance of 78.0%, the testing results demonstrate that the model performs 94.0% better.

show abstract

Section: Related Workmentioning

confidence: 99%

A Convolutional Neural Network Model for Setswana Named Entity Recognition

Chabalala,

Owolawi,

Ojo

2023

iCARTi

View full text Add to dashboard Cite

show abstract

“…Another large set of texts we used was gathered from multiple OPUS corpora [8]. They contain book reviews, subtitles, TED talk transcriptions, etc.…”

Section: Datasetsmentioning

confidence: 99%

Controllability for English-Ukrainian Machine Translation by Using Style Transfer Techniques

Maksymenko,

Saichyshyna,

Paprzycki

et al. 2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

While straightforward machine translation got significant improvements in the last 10 years with the arrival of encoder-decoder neural networks and transformers architecture, controllable machine translation still remains a difficult task, which requires lots of research. Existing methods like tagging provide very limited control over model results or they require to support multiple models at once, like domain fine-tuning approach.In this paper, we propose a method to control translation results style by transferring features from a set of texts with target structure and wording. Our solution consists of new modifications for the encoder-decoder networks, where we can add feature descriptors to each token embedding to decode input text into the translation with the proposed domain. In conducted experiments with English-Ukrainian translation and a set of 4 domains our proposed model gives more options to influence the result than some existing approaches to solve the controllability model.

show abstract

“…All the experiments are done with PyTorch, using the Transformers 4 library and the models were taken from Hugging Face 5 . The transformer models are trained using the original parameters from BERT, presenting a dropout probability for the attention heads and hidden layers of 0.1, a hidden size of 768, an initializer range of 0.02, a max position embeddings of 512 and an intermediate size of 3,072.…”

Section: Parameters Settingsmentioning

confidence: 99%

“…According to different works, such as in [1], solutions to address these issues may include additional features to provide lexical, syntactic and semantic information about text, which have proven to be useful for detecting event triggers. Transformers models have been adopted for event detection due to their positive achievements in performance for solving different Natural Language Processing (NLP) tasks [4], [5]. BERT [6], which stands for Bidirectional Encoder Representations from Transformers, is pretrained to generate bidirectional representations of the words, taking into account the semantics by considering both left and right directions of the text.…”

Section: Introductionmentioning

confidence: 99%