Applying Deep Neural Networks to Named Entity Recognition in Portuguese Texts

Abstract

There is currently few research in using deep learning (DL) applied to Named Entities Recognition (NER) in Portuguese texts. This work exposes some challenges and limitations but also the benefits of applying DL architectures to NER in Portuguese. Four different DL architectures are applied to Portuguese datasets. All architectures are heavily influenced by previous published work in NER applied to English. Annotated data is used to train and test NER models, while non-annotated data is used to train word embeddings, as well as being a key part of a bootstrapping approach, where raw textual data is used to create NER models.

Systematic Literature Review of Information Extraction From Textual Data: Recent Methods, Applications, Trends, and Challenges

“…To reduce errors in the extraction model, the system automatically modifies the connection weights and learning parameters throughout model training. LBMs have also been deployed as state-of-the-art models that feature various DL-based methods for IE from textual data [17], [18], [33], [35], [45], [46], [47].…”

“…Thus, each NER extraction model is limited by the language it is trained to extract [6], [11]. A few examples include NER in Portuguese texts [46], [91], Chinese texts [72], [76], [89], [127], [128], [129], Indonesian texts [65], [78], [87], [119], [130], [143], Malay texts [92], and Arabic texts [21]. This demonstrates that language is a common barrier to extracting useful information from text documents.…”

Uma revisão para o Reconhecimento de Entidades Nomeadas aplicado à língua portuguesa

“…Os word embeddings forneceram uma forma eficiente para a representação de texto em modelos computacionais que só processam informações numéricas, como é o caso das redes neurais. Eles têm sido aplicados não somente em palavras, mas também em caracteres (character embeddings) (Santos & Guimarães, 2015;Fernandes et al, 2018). Além disso, as representações vetoriais podem representar outros tipos de traços, como ortográficos e lexicais.…”

Portuguese Named Entity Recognition using BERT-CRF

“…The LSTM-CRF architecture (Lample et al, 2016) has been commonly used in NER task (Castro et al, 2018;de Araujo et al, 2018;Fernandes et al, 2018). The model is composed of two bidirectional LSTM networks that extract and combine character-level and word-level features.…”

“…A sequential classification is then performed by the CRF layer. Several pre-trained word embeddings were explored by Castro et al (2018) and Fernandes et al (2018) compared it to 3 other architectures. This is the first work to explore a model that benefits from deeper pre-trained language model representations applied to the Portuguese NER task.…”