Scaling Within Document Coreference to Long Texts

Thirukovalluru, Raghuveer; Monath, Nicholas; Shridhar, Kumar; Zaheer, Manzil; Sachan, Mrinmaya; McCallum, Andrew

doi:10.18653/v1/2021.findings-acl.343

Cited by 9 publications

(6 citation statements)

References 16 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table 3 On the other hand, the c2f-coref opt + ELECTRA-large model reaches inferior performance than CorefQA+ SpanBERT-large, but without neither resorting to data augmentation to improve its generalization capability nor processing hundreds of individual context-questionanswer instances for a single document, substantially worsening execution time, as reported by [13]. As further stated in [21], [24], this method has resulted very computationally expensive since it needs to run a transformer-based model to perform a different query on the same document many times. It also exhibiting some difficulties to scale to long documents.…”

Section: A Quantitative Analysismentioning

confidence: 77%

An ELECTRA-Based Model for Neural Coreference Resolution

et al. 2022

View full text Add to dashboard Cite

In last years, coreference resolution has received a sensibly performance boost exploiting different pre-trained Neural Language Models, from BERT to SpanBERT until Longformer. This work is aimed at assessing, for the first time, the impact of ELECTRA model on this task, moved by the experimental evidence of an improved contextual representation and better performance on different downstream tasks. In particular, ELECTRA has been employed as representation layer in an assessed neural coreference architecture able to determine entity mentions among spans of text and to best cluster them. The architecture itself has been optimized: i) by simplifying the modality of representation of spans of text but still considering both the context they appear and their entire content, ii) by maximizing both the number and length of input textual segments to exploit better the improved contextual representation power of ELECTRA, iii) by maximizing the number of spans of text to be processed, since potentially representing mentions, preserving computational efficiency. Experimental results on the OntoNotes dataset have shown the effectiveness of this solution from both a quantitative and qualitative perspective, and also with respect to other state-of-the-art models, thanks to a more proficient token and span representation. The results also hint at the possible use of this solution also for low-resource languages, simply requiring a pre-trained version of ELECTRA instead of language-specific models trained to handle either spans of text or long documents.

show abstract

Section: A Quantitative Analysismentioning

confidence: 77%

An ELECTRA-Based Model for Neural Coreference Resolution

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Much of the recent work on coreference can be organized into three categories: span based representations (Lee et al, 2017;Joshi et al, 2020), token-wise representations (Thirukovalluru et al, 2021;Kirstain et al, 2021) and memory networks / incremental models (Toshniwal et al, 2020b,a). We consider one approach from all three categories.…”

Section: Modelsmentioning

confidence: 99%

“…Our proposed dataset, LongtoNotes, restores documents to their original form, revealing dramatic increases in length in certain genres. Sachan et al, 2015;Wiseman et al, 2016;Lee et al, 2017;Joshi et al, 2020;Toshniwal et al, 2020b;Thirukovalluru et al, 2021;Kirstain et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Longtonotes: OntoNotes with Longer Coreference Chains

Shridhar,

Monath,

Thirukovalluru

et al. 2023

Findings of the Association for Computational Linguistics: EACL 2023

View full text Add to dashboard Cite

Ontonotes has served as the most important benchmark for coreference resolution. However, for ease of annotation, long documents in Ontonotes were split into smaller parts. In this work, we build a corpus of coreferenceannotated documents of significantly longer length than what is currently available. We do so by providing an accurate, manuallycurated, merging of annotations from documents that were split into multiple parts in the original Ontonotes annotation process (Pradhan et al., 2013). The resulting corpus, which we call LongtoNotes contains documents in multiple genres of the English language with varying lengths, the longest of which are up to 8x the length of documents in Ontonotes, and 2x those in Litbank. We evaluate stateof-the-art neural coreference systems on this new corpus, analyze the relationships between model architectures/hyperparameters and document length on performance and efficiency of the models, and demonstrate areas of improvement in long-document coreference modelling revealed by our new corpus. Our data and code is available at: https://github. com/kumar-shridhar/LongtoNotes.

show abstract

“…Swayamdipta et al (2018) also leverage syntactic span classification as an auxiliary task to assist coreference. Thirukovalluru et al (2021), Kirstain et al (2021), andDobrovolskii (2021) explore token-level representations to both reduce memory consumption and increase performance on longer documents. Miculicich and Henderson (2020) and Yu et al (2020) both improve the mention detector with better neural network structures.…”

Section: Coreference Resolutionmentioning

confidence: 99%

A Structured Span Selector

Li¹,

Jiang²,

Cotterell³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

Many natural language processing tasks, e.g., coreference resolution and semantic role labeling, require selecting text spans and making decisions about them. A typical approach to such tasks is to score all possible spans and greedily select spans for task-specific downstream processing. This approach, however, does not incorporate any inductive bias about what sort of spans ought to be selected, e.g., that selected spans tend to be syntactic constituents. In this paper, we propose a novel grammar-based structured span selection model which learns to make use of the partial span-level annotation provided for such problems. Compared to previous approaches, our approach gets rid of the heuristic greedy span selection scheme, allowing us to model the downstream task on an optimal set of spans. We evaluate our model on two popular span prediction tasks: coreference resolution and semantic role labeling. We show empirical improvements on both. https://github.com/lyutyuh/ structured-span-selector

show abstract

Scaling Within Document Coreference to Long Texts

Cited by 9 publications

References 16 publications

An ELECTRA-Based Model for Neural Coreference Resolution

An ELECTRA-Based Model for Neural Coreference Resolution

Longtonotes: OntoNotes with Longer Coreference Chains

A Structured Span Selector

Contact Info

Product

Resources

About