Named Entity Recognition Only from Word Embeddings

Abstract

Deep neural network models have helped named entity (NE) recognition achieve amazing performance without handcrafting features. However, existing systems require large amounts of human annotated training data. Efforts have been made to replace human annotations with external knowledge (e.g., NE dictionary, part-of-speech tags), while it is another challenge to obtain such effective resources. In this work, we propose a fully unsupervised NE recognition model which only needs to take informative clues from pre-trained word embeddings. We first apply Gaussian Hidden Markov Model and Deep Autoencoding Gaussian Mixture Model on word embeddings for entity span detection and type prediction, and then further design an instance selector based on reinforcement learning to distinguish positive sentences from noisy sentences and refine these coarse-grained annotations through neural networks. Extensive experiments on CoNLL benchmark datasets demonstrate that our proposed light NE recognition model achieves remarkable performance without using any annotated lexicon or corpus.

Genealogical Data-Driven Visits of Historical Cemeteries

“…Some works have also performed NER without labeled data or pretrained models. For example, Luo et al propose a fully unsupervised NER model that only relies on pretrained word embeddings [17]. Another approach uses natural language prompts to guide LLMs to perform NER without fine-tuning or labeling data [18].…”

Learning workload behaviour models from monitored time-series for resource estimation towards data center optimization

“…Learning a vector representation is, in many cases, a preprocessing step to facilitate another task. For example, the vector representation learned by Word2Vec [52] has been used to map words to dense feature vectors to solve tasks such as Named Entity Recognition [38,48].…”