Deep Affix Features Improve Neural Named Entity Recognizers

Yadav, Vikas; Sharp, Rebecca; Bethard, Steven

doi:10.18653/v1/s18-2021

Cited by 68 publications

(76 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text De-identification system architecture Our machine learning model implements the state-of-the-art in de-identification of medical notes [5,6] and named entity sequence tagging [30]. In our analyses, however, any sufficiently powerful model could be substituted.…”

Section: Data Sourcesmentioning

confidence: 99%

Customization scenarios for de-identification of clinical notes

Hartman

Howell

Dean

et al. 2020

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background: Automated machine-learning systems are able to de-identify electronic medical records, including free-text clinical notes. Use of such systems would greatly boost the amount of data available to researchers, yet their deployment has been limited due to uncertainty about their performance when applied to new datasets. Objective: We present practical options for clinical note de-identification, assessing performance of machine learning systems ranging from off-the-shelf to fully customized. Methods: We implement a state-of-the-art machine learning de-identification system, training and testing on pairs of datasets that match the deployment scenarios. We use clinical notes from two i2b2 competition corpora, the Physionet Gold Standard corpus, and parts of the MIMIC-III dataset. Results: Fully customized systems remove 97-99% of personally identifying information. Performance of off-the-shelf systems varies by dataset, with performance mostly above 90%. Providing a small labeled dataset or large unlabeled dataset allows for fine-tuning that improves performance over off-the-shelf systems. Conclusion: Health organizations should be aware of the levels of customization available when selecting a deidentification deployment solution, in order to choose the one that best matches their resources and target performance level.

show abstract

Section: Data Sourcesmentioning

confidence: 99%

Customization scenarios for de-identification of clinical notes

Hartman

Howell

Dean

et al. 2020

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…This model became the state-of-the-art system (85.81%) for Spanish NER until 2018. Finally, the most similar architecture towards ours proposed by Yadav et al in 2018 [25]. They combined affix-level features along with frequently explored word + character models.…”

Section: Related Workmentioning

confidence: 92%

“…Instead, we consider affixes at both ends of the word as additional features for NER. Our base model is similar to Reference [25], where we apply the combined representation of character-level, word-level and affix-level features over Bi-LSTM-CRF stack. Our model differs from their architecture in how the character-level and affix-level features are generated.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…In this paper, we further explore an effective representation for words in Indian languages through a combination of character-level, word-level and affix-level embeddings. In our approach, we augment the methodology used by Vikas Yadav et al in 2018 [25]. The only difference is in the way in which character-based word vectors are generated and frequent affixes are identified for each language.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Improved Word Representation for Deep Learning Based NER in Indian Languages

2019

View full text Add to dashboard Cite

Named Entity Recognition (NER) is the process of identifying the elementary units in a text document and classifying them into predefined categories such as person, location, organization and so forth. NER plays an important role in many Natural Language Processing applications like information retrieval, question answering, machine translation and so forth. Resolving the ambiguities of lexical items involved in a text document is a challenging task. NER in Indian languages is always a complex task due to their morphological richness and agglutinative nature. Even though different solutions were proposed for NER, it is still an unsolved problem. Traditional approaches to Named Entity Recognition were based on the application of hand-crafted features to classical machine learning techniques such as Hidden Markov Model (HMM), Support Vector Machine (SVM), Conditional Random Field (CRF) and so forth. But the introduction of deep learning techniques to the NER problem changed the scenario, where the state of art results have been achieved using deep learning architectures. In this paper, we address the problem of effective word representation for NER in Indian languages by capturing the syntactic, semantic and morphological information. We propose a deep learning based entity extraction system for Indian languages using a novel combined word representation, including character-level, word-level and affix-level embeddings. We have used ‘ARNEKT-IECSIL 2018’ shared data for training and testing. Our results highlight the improvement that we obtained over the existing pre-trained word representations.

show abstract

“…NER is a widely studied problem, where methods have been characterized by the use of CRFs with heavy feature engineering, gazetteers and external knowledge resources (Finkel et al, 2005;Florian et al, 2003;Kazama and Torisawa, 2007;Klein et al, 2003;Lin and Wu, 2009;Radford et al, 2015;Ratinov and Roth, 2009;Zhang and Johnson, 2003). Ratinov and Roth (2009) (Bharadwaj et al, 2016;Chiu and Nichols, 2016;Collobert et al, 2011;Gillick et al, 2016;Huang et al, 2015;Lample et al, 2016;Ma and Hovy, 2016;dos Santos and Guimarães, 2015;Yadav et al, 2018;Yang et al, 2016). Huang et al (2015) use a word-level bidirectional LSTM-CRF for several sequence tagging problems including POS tagging and NER.…”

Section: Related Workmentioning

confidence: 99%

Neural Named Entity Recognition from Subword Units

Abujabal

Gaspers

2019

Interspeech 2019

View full text Add to dashboard Cite

Named entity recognition (NER) is a vital task in language technology. Existing neural models for NER rely mostly on dedicated word-level representations, which suffer from two main shortcomings: 1) the vocabulary size is large, yielding large memory requirements and training time, and 2) they cannot learn morphological representations. We adopt a neural solution based on bidirectional LSTMs and conditional random fields, where we rely on subword units, namely characters, phonemes, and bytes, to remedy the above shortcomings. We conducted experiments on a large dataset covering four languages with up to 5.5M utterances per language. Our experiments show that 1) with increasing training data, performance of models trained solely on subword units becomes closer to that of models with dedicated word-level embeddings (91.35 vs 93.92 F1 for English), while using a much smaller vocabulary size (332 vs 74K), 2) subword units enhance models with dedicated word-level embeddings, and 3) combining different subword units improves performance.

show abstract

Deep Affix Features Improve Neural Named Entity Recognizers

Cited by 68 publications

References 18 publications

Customization scenarios for de-identification of clinical notes

Customization scenarios for de-identification of clinical notes

An Improved Word Representation for Deep Learning Based NER in Indian Languages

Neural Named Entity Recognition from Subword Units

Contact Info

Product

Resources

About