Transfer Learning for Arabic Named Entity Recognition With Deep Neural Networks

Al-Smadi, Mohammad; Al-Zboon, Sa’ad A.; Jararweh, Yaser; Juola, Patrick

doi:10.1109/access.2020.2973319

Cited by 41 publications

(20 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ali and Tan [64] used seq2seq model with BLSTM as an encoder and decoder model for MSA NER; their model outperformed the BGRU-CRF model by [61] and the BLSTM-CRF model by [57]. A recent study [65] applied transfer learning with deep neural networks to build a Pooled-GRU model for MSA NER. Their model outperformed the BLSTM-CRF model proposed by [66].…”

Section: Related Workmentioning

confidence: 99%

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Alsaaran

Alrabiah

2021

IEEE Access

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) and transformers are deep learning models that have achieved remarkable success in several Natural Language Processing (NLP) tasks since they do not rely on handcrafted features nor enormous knowledge resources. Named Entity Recognition (NER) is an essential NLP task that is used in many applications such as information retrieval, question answering, and machine translation. NER aims to locate, extract, and classify named entities into predefined categories such as person, organization and location. Arabic NER is considered a challenging task because of the complexity and the unique characteristics of Arabic. Most of the previous research on deep learning based-Arabic NER focused on Modern Standard Arabic and Dialectal Arabic, which are different variations from Classical Arabic. In this paper, we investigate deep learning-based Classical Arabic NER using different deep neural network architectures and a BERT based contextual language model that is trained on general domain Arabic text. We propose two RNN-based models by fine-tunning the pretrained BERT language model to recognize and classify named entities from Classical Arabic. The pre-trained BERT contextual language model representations were used as input features to a BGRU/BLSTM model and were fine-tuned using a Classical Arabic NER dataset. In addition, we explore variant architectures of the proposed BERT-BGRU/BLSTM-CRF models. Experimentations showed that the BERT-BGRU-CRF model outperformed the other models by achieving an F-measure of 94.76% on the CANERCorpus. To the best of our knowledge, this is the first work that aims to recognize named entities in Classical Arabic using deep learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Alsaaran

Alrabiah

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Language models and transfer learning have recently significantly contributed to this field. Transfer learning increases the performances of CR applications, as seen in [25,26], requiring a smaller labelled dataset than training from scratch. Furthermore, the contextualised representation contributes to recognising and differentiating concepts based on their context, thus increasing the accuracy of the model predictions.…”

Section: Concept Recognition For Space Systemsmentioning

confidence: 99%

SpaceTransformers: Language Modeling for Space Systems

Berquand¹,

Darm

Riccardi

2021

IEEE Access

View full text Add to dashboard Cite

The transformers architecture and transfer learning have radically modified the Natural Language Processing (NLP) landscape, enabling new applications in fields where open source labelled datasets are scarce. Space systems engineering is a field with limited access to large labelled corpora and a need for enhanced knowledge reuse of accumulated design data. Transformers models such as the Bidirectional Encoder Representations from Transformers (BERT) and the Robustly Optimised BERT Pretraining Approach (RoBERTa) are however trained on general corpora. To answer the need for domainspecific contextualised word embedding in the space field, we propose SpaceTransformers, a novel family of three models, SpaceBERT, SpaceRoBERTa and SpaceSciBERT, respectively further pre-trained from BERT, RoBERTa and SciBERT on our domain-specific corpus. We collect and label a new dataset of space systems concepts based on space standards. We fine-tune and compare our domain-specific models to their general counterparts on a domain-specific Concept Recognition (CR) task. Our study rightly demonstrates that the models further pre-trained on a space corpus outperform their respective baseline models in the Concept Recognition task, with SpaceRoBERTa achieving significant higher ranking overall.

show abstract

“…More modern neural network architectures based on various combinations of convolutional and recurrent networks show the best results in solving many problems [31][32][33][34]. These models show significant versatility because they can be applied to multiple languages with unified network architecture.…”

Section: ♦ Improving the Means Of Monitoring Open Sources;mentioning

confidence: 99%

An approach to identifying threats of extracting confidential data from automated control systems based on internet technologies

Kuzmin¹,

Menisov²

2021

Bus. Inform.

View full text Add to dashboard Cite

Together with ubiquitous, global digitalization, cybercrime is growing and developing rapidly. The state considers the creation of an environment conducive to information security to be a strategic goal for the development of the information society in Russia. However, the question of how the “state of protection of the individual, society and the state from internal and external information threats” should be achieved in accordance with the “Information Security” and the “Digital Economy of Russia 2024” programs remains open. The aim of this study is to increase the efficiency whereby automated control systems identify confidential data from html-pages to reduce the risk of using this data in the preparatory and initial stages of attacks on the infrastructure of government organizations. The article describes an approach that has been developed to identify confidential data based on the combination of several neural network technologies: a universal sentence encoder and a neural network recurrent architecture of bidirectional long-term short-term memory. The results of an assessment in comparison with modern means of natural language text processing (SpaCy) showed the merits and prospects of the practical application of the methodological approach.

show abstract

Transfer Learning for Arabic Named Entity Recognition With Deep Neural Networks

Cited by 41 publications

References 39 publications

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

SpaceTransformers: Language Modeling for Space Systems

An approach to identifying threats of extracting confidential data from automated control systems based on internet technologies

Contact Info

Product

Resources

About