A Structured Span Selector

Li, Tianyu; Jiang, Yuchen; Cotterell, Ryan; Sachan, Mrinmaya

doi:10.18653/v1/2022.naacl-main.189

Cited by 8 publications

(14 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Table 4, Bohnet and others [26] and Liu and others [25] reported average F1 scores of 83.3 and 82.3 for CR, respectively. However, to achieve such a high performance, they used mT5 XXL (LM with a parameter size of 13 billion) and T0 3B (LM with a parameter size of 3 billion).…”

Section: Resultsmentioning

confidence: 94%

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

Jung

2024

ETRI Journal

View full text Add to dashboard Cite

This study introduces CR‐M‐SpanBERT, a coreference resolution (CR) model that utilizes multiple embedding‐based span bidirectional encoder representations from transformers, for antecedent recognition in natural language (NL) text. Information extraction studies aimed to extract knowledge from NL text autonomously and cost‐effectively. However, the extracted information may not represent knowledge accurately owing to the presence of ambiguous entities. Therefore, we propose a CR model that identifies mentions referring to the same entity in NL text. In the case of CR, it is necessary to understand both the syntax and semantics of the NL text simultaneously. Therefore, multiple embeddings are generated for CR, which can include syntactic and semantic information for each word. We evaluate the effectiveness of CR‐M‐SpanBERT by comparing it to a model that uses SpanBERT as the language model in CR studies. The results demonstrate that our proposed deep neural network model achieves high‐recognition accuracy for extracting antecedents from NL text. Additionally, it requires fewer epochs to achieve an average F1 accuracy greater than 75% compared with the conventional SpanBERT approach.

show abstract

Section: Resultsmentioning

confidence: 94%

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

Jung

2024

ETRI Journal

View full text Add to dashboard Cite

show abstract

“…On ACE 05, ATG provide a competitive performance, securing the second-highest scores. The reported top-performing model, ASP (Liu et al 2022), operates under a relaxed, undirected relation evaluation, thereby limiting a fair comparison of results (Taillé et al 2021). On the CoNLL 2004 dataset, ATG exhibits its superiority by outperforming the second-best result by 2.2 in terms of REL+.…”

Section: Resultsmentioning

confidence: 99%

“…Recent advancements in generative Information Extraction (IE) emphasize the use of language models (LMs) to produce entities and relations, either as text or as a sequence of actions (Paolini et al 2021;Lu et al 2022;Nayak and Ng 2020;Liu et al 2022;Fei et al 2022;Wan et al 2023). Typically, these models employ pretrained encoder-decoder architectures, such as T5 (Raffel et al 2019) or BART (Lewis et al 2020), to encode an input text and subsequently decode it into a structured output.…”

Section: Generative Iementioning

confidence: 99%

“…Another solution is to build a custom decoder that is constrained to a tailored vocabulary and decoding algorithm guaranteed to produce a linearized wellformed structure. Liu et al (2022) and Lu et al (2022) use a specialized decoder with some sort of explicit grammar over a specific vocabulary to restrict the output to valid sequences. Our work falls in this category which can be formalized as generating words from a formal language described by a grammar over a specialized alphabet (Willard and Louf 2023;Geng et al 2023).…”

Section: Constrained Decodingmentioning

confidence: 99%

“…Recent advancements have seen a shift towards "real" end-to-end solutions, where the prediction of entities and relations is intertwined, accomplished through autoregressive models. These models treat the joint entity-relation task as a process of generating plain text, employing augmented languages to encode and decode structural information (Paolini et al 2021;Lu et al 2022;Liu et al 2022;Fei et al 2022).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

Zaratiana,

Tomeh,

Holat

et al. 2024

AAAI

View full text Add to dashboard Cite

In this paper, we propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. In contrast to conventional generative information extraction models that are left-to-right token-level generators, our approach is \textit{span-based}. It generates a linearized graph where nodes represent text spans and edges represent relation triplets. Our method employs a transformer encoder-decoder architecture with pointing mechanism on a dynamic vocabulary of spans and relation types. Our model can capture the structural characteristics and boundaries of entities and relations through span representations while simultaneously grounding the generated output in the original text thanks to the pointing mechanism. Evaluation on benchmark datasets validates the effectiveness of our approach, demonstrating competitive results. Code is available at https://github.com/urchade/ATG.

show abstract

Sentence-Incremental Neural Coreference Resolution

Grenander¹,

Cohen²,

Steedman³

2022

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

We propose a sentence-incremental neural coreference resolution system which incrementally builds clusters after marking mention boundaries in a shift-reduce method. The system is aimed at bridging two recent approaches at coreference resolution: (1) state-of-the-art non-incremental models that incur quadratic complexity in document length with high computational cost, and (2) memory networkbased models which operate incrementally but do not generalize beyond pronouns. For comparison, we simulate an incremental setting by constraining non-incremental systems to form partial coreference chains before observing new sentences. In this setting, our system outperforms comparable state-of-the-art methods by 2 F1 on OntoNotes and 6.8 F1 on the CODI-CRAC 2021 corpus. In a conventional coreference setup, our system achieves 76.3 F1 on OntoNotes and 45.5 F1 on CODI-CRAC 2021, which is comparable to state-of-the-art baselines. We also analyze variations of our system and show that the degree of incrementality in the encoder has a surprisingly large effect on the resulting performance. 1

show abstract

A Structured Span Selector

Cited by 8 publications

References 62 publications

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

Sentence-Incremental Neural Coreference Resolution

Contact Info

Product

Resources

About