Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Abstract

This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off. We present a two-stage zero-shot linking algorithm, where each entity is defined only by a short textual description. The first stage does retrieval in a dense space defined by a bi-encoder that independently embeds the mention context and the entity descriptions. Each candidate is then re-ranked with a crossencoder, that concatenates the mention and entity text. Experiments demonstrate that this approach is state of the art on recent zeroshot benchmarks (6 point absolute gains) and also on more established non-zero-shot evaluations (e.g. TACKBP-2010), despite its relative simplicity (e.g. no explicit entity embeddings or manually engineered mention tables). We also show that bi-encoder linking is very fast with nearest neighbour search (e.g. linking with 5.9 million candidates in 2 milliseconds), and that much of the accuracy gain from the more expensive crossencoder can be transferred to the bi-encoder via knowledge distillation. Our code and models are available at https://github. com/facebookresearch/BLINK.

Generating Fact Checking Briefs

“…Entities in each claim are identified with BLINK (Wu et al, 2019), a model trained on Wikipedia data that links each entity to its nearest Wikipedia page. BLINK combines a bi-encoder (Urbanek et al, 2019;) that identifies candidates with a cross-encoder that models the interaction between mention context and entity descriptions.…”

Entity Linking in 100 Languages

“…We observed improvements on most frequency buckets compared to DE R@1, which suggests that the model's few-shot capability can be improved by cross-lingual reading-comprehension. This also offers an initial multilingual validation of a similar two-step BERT-based approach recently introduced in a monolingual setting by (Wu et al, 2019), and provides a strong baseline for future work.…”

“…Basing entity representations on features of their Wikipedia pages has been a common approach in EL (e.g. Sil and Florian, 2016;Francis-Landau et al, 2016;Gillick et al, 2019;Wu et al, 2019), but we will need to generalize this to include multiple Wikipedia pages with possibly redundant features in many languages.…”

“…Prior work showed that a dual encoder architecture can encode entities and contextual mentions in a dense vector space to facilitate efficient entity retrieval via nearest-neighbors search (Gillick et al, 2019;Wu et al, 2019). We take the same approach.…”

TrainX – Named Entity Linking with Active Sampling and Bi-Encoders

“…By using active sampling, we minimize labeling efforts. TrainX uses transfer learning by leveraging Bi-Encoders (Gillick et al, 2019;Wu et al, 2019;Logeswaran et al, 2019;Humeau et al, 2020) for disambiguation and a kNN-index to retrieve candidate entities within milliseconds. We mitigate issues caused by sparse training data by using zero-shot optimized techniques that can generalize beyond the labels seen in training.…”