Measuring and Improving Consistency in Pretrained Language Models

Elazar, Yanai; Kassner, Nora; Ravfogel, Shauli; Ravichander, Abhilasha; Hovy, Eduard; Schütze, Hinrich; Goldberg, Yoav

doi:10.1162/tacl_a_00410

Cited by 95 publications

(113 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, constructing prompts from supervised knowledge extraction data risks learning new knowledge instead of recalling existing knowledge in an LM (Zhong et al, 2021). More recently, Elazar et al (2021a) introduced ParaRel, a curated dataset of paraphrased prompts and facts. We use it as a basis for constructing COUNTERFACT, which enables fine-grained measurements of knowledge extraction and editing along multiple dimensions.…”

Section: Extracting Knowledge From Lmsmentioning

confidence: 99%

“…Despite increasing adoption of this architecture, their knowledge representation remains under-explored. Research has been done for masked models (Petroni et al, 2019;Jiang et al, 2020;Elazar et al, 2021a;Geva et al, 2021;Dai et al, 2021;De Cao et al, 2021), but GPT's architectural differences (e.g., unidirectional attention, generation capabilities) provide an opportunity for new insights.…”

Section: Introductionmentioning

confidence: 99%

“…Existing Evaluation FrameworksC.1. Compilation MethodologyEach record in COUNTERFACT is derived from a corresponding entry in PARAREL(Elazar et al, 2021a) containing a knowledge tuple t c = (s, r, o c ) and hand-curated prompt templates T (r). Notice that prompt templates are unique only to relations; entities can be plugged in to form full prompts: P(s, r) {t.format(s) | t ∈ T (r)}, where .format() is syntax for string substitution.7 Solely using the PARAREL entry, we derive two elements.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Locating and Editing Factual Associations in GPT

Meng¹,

Bau²,

Andonian³

et al. 2022

Preprint

View full text Add to dashboard Cite

We investigate the mechanisms underlying factual knowledge recall in autoregressive transformer language models. First, we develop a causal intervention for identifying neuron activations capable of altering a model's factual predictions. Within large GPT-style models, this reveals two distinct sets of neurons that we hypothesize correspond to knowing an abstract fact and saying a concrete word, respectively. This insight inspires the development of ROME, a novel method for editing facts stored in model weights. For evaluation, we assemble COUNTERFACT, a dataset of over twenty thousand counterfactuals and tools to facilitate sensitive measurements of knowledge editing. Using COUNTERFACT, we confirm the distinction between saying and knowing neurons, and we find that ROME achieves state-of-the-art performance in knowledge editing compared to other methods. An interactive demo notebook, full code implementation, and the dataset are available at https://rome.baulab.info/.

show abstract

Section: Extracting Knowledge From Lmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Locating and Editing Factual Associations in GPT

Meng¹,

Bau²,

Andonian³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As large-scale language models are gradually evolving towards more abstract inference, it is crucial to study and understand the underlying semantics encoded in their representation to identify biases and inconsistencies within the models (Elazar et al, 2021a), improve transparency (Thayaparan et al, 2020), and further investigate their generalisation and reasoning capabilities (Hu et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Do Transformers Encode a Foundational Ontology? Probing Abstract Classes in Natural Language

Jullien¹,

Valentino²,

Freitas³

2022

Preprint

View full text Add to dashboard Cite

With the methodological support of probing (or diagnostic classification), recent studies have demonstrated that Transformers encode syntactic and semantic information to some extent. Following this line of research, this paper aims at taking semantic probing to an abstraction extreme with the goal of answering the following research question: can contemporary Transformer-based models reflect an underlying Foundational Ontology? To this end, we present a systematic Foundational Ontology (FO) probing methodology to investigate whether Transformers-based models encode abstract semantic information. Following different pre-training and fine-tuning regimes, we present an extensive evaluation of a diverse set of large-scale language models over three distinct and complementary FO tagging experiments. Specifically, we present and discuss the following conclusions: (1) The probing results indicate that Transformer-based models incidentally encode information related to Foundational Ontologies during the pre-training process; (2) Robust FO taggers (accuracy ≈ 90% ) can be efficiently built leveraging on this knowledge.

show abstract

“…But beyond the intuition that patterns serve as some sort of task instruction (Schick and Schütze, 2021a), little is known about the reasons for their success. Recent findings that (i) PLMs can fail to follow even simple instructions (Efrat and Levy, 2020), that (ii) PLMs can behave drastically different with paraphrases of the same pattern (Elazar et al, 2021), and that (iii) performance increases if we train a second model to rewrite an input pattern with the goal of making it more comprehensible for a target PLM (Haviv et al, 2021), strongly suggest that patterns do not make sense to PLMs in the same way as they do to humans.…”

Section: Introductionmentioning

confidence: 99%

Continuous Entailment Patterns for Lexical Inference in Context

Schmitt¹,

Schütze²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Combining a pretrained language model (PLM) with textual patterns has been shown to help in both zero-and few-shot settings. For zero-shot performance, it makes sense to design patterns that closely resemble the text seen during self-supervised pretraining because the model has never seen anything else. Supervised training allows for more flexibility. If we allow for tokens outside the PLM's vocabulary, patterns can be adapted more flexibly to a PLM's idiosyncrasies. Contrasting patterns where a "token" can be any continuous vector vs. those where a discrete choice between vocabulary elements has to be made, we call our method CONtinuous pAt-terNs (CONAN). We evaluate CONAN on two established benchmarks for lexical inference in context (LIiC) a.k.a. predicate entailment, a challenging natural language understanding task with relatively small training sets. In a direct comparison with discrete patterns, CO-NAN consistently leads to improved performance, setting a new state of the art. Our experiments give valuable insights into the kind of pattern that enhances a PLM's performance on LIiC and raise important questions regarding our understanding of PLMs using text patterns. 1

show abstract

Measuring and Improving Consistency in Pretrained Language Models

Cited by 95 publications

References 55 publications

Locating and Editing Factual Associations in GPT

Locating and Editing Factual Associations in GPT

Do Transformers Encode a Foundational Ontology? Probing Abstract Classes in Natural Language

Continuous Entailment Patterns for Lexical Inference in Context

Contact Info

Product

Resources

About