Modeling Fine-Grained Entity Types with Box Embeddings

Abstract

Neural entity typing models typically represent entity types as vectors in a highdimensional space, but such spaces are not well-suited to modeling these types' complex interdependencies. We study the ability of box embeddings, which represent entity types as ddimensional hyperrectangles, to represent hierarchies of fine-grained entity type labels even when these relationships are not defined explicitly in the ontology. Our model represents both types and entity mentions as boxes. Each mention and its context are fed into a BERT-based model to embed that mention in our box space; essentially, this model leverages typological clues present in the surface text to hypothesize a type representation for the mention. Soft box containment can then be used to derive probabilities, both the posterior probability of a mention exhibiting a given type and the conditional probability relations between types themselves. We compare our approach with a strong vector-based typing model, and observe state-of-the-art performance on several entity typing benchmarks. In addition to competitive typing performance, our box-based model shows better performance in prediction consistency (predicting a supertype and a subtype together) and confidence (i.e., calibration), implying that the box-based model captures the latent type hierarchies better than the vectorbased model does.

Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

“…Our approach achieves the best F1 score. It obtains more than 4% F1 score improvement over the existing best reported performance by Box in (Onoe et al, 2021). This demonstrates the effectiveness of our approach.…”

“…A graph propagation layer is introduced by (Xiong et al, 2019) to impose a label-relational bias on entity typing models, so as to implicitly capture type dependencies. Onoe et al (2021) use box embeddings to capture latent type hierarchies.…”

“…• Box (Onoe et al, 2021). Box represents entity types with box embeddings to capture latent type hierarchies.…”

“…Adding any more patterns cannot improve the F1 score for more than 0.007. Following existing work (Onoe et al, 2021;Onoe and Durrett, 2019), we evaluate the macroaveraged precision, recall, and F1 of different approaches on the manually annotated test set. The results are in Table 3.…”

Learning from Sibling Mentions with Scalable Graph Inference in Fine-Grained Entity Typing

“…Chen et al (2020) designs a multi-level learning-to-rank loss to leverage hierarchical information. Recently, Onoe et al (2021) models the mention and type representations in a box space instead of the traditional vector space.…”