Transfer learning for Turkish named entity recognition on noisy text

Abstract

In this article, we investigate using deep neural networks with different word representation techniques for named entity recognition (NER) on Turkish noisy text. We argue that valuable latent features for NER can, in fact, be learned without using any hand-crafted features and/or domain-specific resources such as gazetteers and lexicons. In this regard, we utilize character-level, character n-gram-level, morpheme-level, and orthographic character-level word representations. Since noisy data with NER annotation are scarce for Turkish, we introduce a transfer learning model in order to learn infrequent entity types as an extension to the Bi-LSTM-CRF architecture by incorporating an additional conditional random field (CRF) layer that is trained on a larger (but formal) text and a noisy text simultaneously. This allows us to learn from both formal and informal/noisy text, thus improving the performance of our model further for rarely seen entity types. We experimented on Turkish as a morphologically rich language and English as a relatively morphologically poor language. We obtained an entity-level F1 score of 67.39% on Turkish noisy data and 45.30% on English noisy data, which outperforms the current state-of-art models on noisy text. The English scores are lower compared to Turkish scores because of the intense sparsity in the data introduced by the user writing styles. The results prove that using subword information significantly contributes to learning latent features for morphologically rich languages.

Generative adversarial network and transfer-learning-based fault detection for rotating machinery with imbalanced data condition

“…Transfer learning refers to a learning process that uses existing knowledge and similarity between data and models to solve problems in different but related domains [26]. This technique can solve the problem of sparse training samples in deep learning, and has been successfully applied in the fields of image recognition [27,28], speech recognition [29,30], and text recognition [31,32]. Transfer learning is a machine-learning paradigm that generalizes the commonalities between different tasks or domains and applies the models from the source domain into the target domain.…”

TurkishDelightNLP: A Neural Turkish NLP Toolkit

“…We use a BiLSTM-CRF model where each word is encoded through a BiLSTM and decoded with a CRF layer to learn the named entities in a given text (Kagan Akkaya and Can, 2021). We feed the BiLSTM with character-level (learned through a character-level BiLSTM), character n-gram-level (fasttext), morpheme-level (morph2vec), and wordlevel word embeddings (word2vec), as well as orthographic embeddings that are learned either with a CNN or BiLSTM by encoding alphabetic characters similar to that of Aguilar et al (2017).…”

Named-entity recognition in Turkish legal texts

“…They reached an F1 score of 91.30% without using any external resources. Akkaya and Can (2021) tackled Turkish NER problem with transfer learning and additional CRF layers reaching an entity level F1 score of 67.39% for a non-domain-specific noisy corpus.…”