Pushing the Limits of AMR Parsing with Self-Learning

Lee, Young-Suk; Astudillo, Ramón Fernandez; Naseem, Tahira; Reddy, Revanth Gangi; Florian, Radu; Roukos, Salim

doi:10.18653/v1/2020.findings-emnlp.288

Cited by 21 publications

(23 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our small model only trails the base model by a small margin and we achieve high performance on small AMR 1.0 dataset, indicating that our approach benefits from having good inductive bias towards the problem so that the learning is efficient. More remarkably, we even surpass the scores reported in Lee et al (2020) combining various self-learning techniques and utilizing 85K extra sentences for self-annotation (silver data). For the most recent AMR 3.0 dataset, we report our results for future reference.…”

Section: Resultscontrasting

confidence: 59%

“…We thus use this ensemble to annotate the 85K sentence set used in (Lee et al, 2020). After removing parses with detached nodes we obtained 70K model-annotated silver data sentences.…”

Section: Resultsmentioning

confidence: 99%

“…For graph embedding, we use 2 heads of the bottom 3 layers for the base model and bottom 2 layers for the small model. We use contextualized embeddings extracted from the pre-trained RoBERTa large model for the source sentence, with average of all layer states and BPE tokens mapped to words by averaging as in (Lee et al, 2020). The pre-trained embeddings are fixed.…”

Section: Model Configurationmentioning

confidence: 99%

“…Transition-based approaches predict a sequence of actions given the sentence. These actions generate the graph while processing tokens left-to-right through the sentence and store intermediate representations in memories such as stack and buffer (Wang et al, 2015;Damonte et al, 2016;Ballesteros and Al-Onaizan, 2017;Vilares and Gómez-Rodríguez, 2018;Naseem et al, 2019;Lee et al, 2020). General graph-based approaches, on the other hand, directly predict nodes and edges in sequential order from graph traversals such as breath first search or depth first search (Zhang et al, 2019a,b;Lam, 2019, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

AMR Parsing with Action-Pointer Transformer

Zhou

Naseem

Astudillo

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

Meaning Representation parsing is a sentence-to-graph prediction task where target nodes are not explicitly aligned to sentence tokens. However, since graph nodes are semantically based on one or more sentence tokens, implicit alignments can be derived. Transition-based parsers operate over the sentence from left to right, capturing this inductive bias via alignments at the cost of limited expressiveness. In this work, we propose a transition-based system that combines hard-attention over sentences with a targetside action pointer mechanism to decouple source tokens from node representations and address alignments. We model the transitions as well as the pointer mechanism through straightforward modifications within a single Transformer architecture. Parser state and graph structure information are efficiently encoded using attention heads. We show that our action-pointer approach leads to increased expressiveness and attains large gains (+1.6 points) against the best transition-based AMR parser in very similar conditions. While using no graph re-categorization, our single model yields the second best SMATCH score on AMR 2.0 (81.8), which is further improved to 83.4 with silver data and ensemble decoding. I offer a solution to the problem .

show abstract

Section: Resultscontrasting

confidence: 59%

“…We thus use this ensemble to annotate the 85K sentence set used in (Lee et al, 2020). After removing parses with detached nodes we obtained 70K model-annotated silver data sentences.…”

Section: Resultsmentioning

confidence: 99%

Section: Model Configurationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AMR Parsing with Action-Pointer Transformer

Zhou

Naseem

Astudillo

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

show abstract

“…Xu et al (2020a) improved sequence-to-sequence parsing for AMR by using pre-trained encoders, reaching similar performance to Cai and Lam (2020). introduced a stack-transformer to enhance transitionbased AMR parsing (Ballesteros and Al-Onaizan, 2017), and Lee et al (2020) improved it further, using a trained parser for mining oracle actions and combining it with AMR-to-text generation to outperform the state of the-art. Wang et al (2018) parsed Chinese AMR with a transition-based system.…”

Section: Overview Of Approachesmentioning

confidence: 93%

MRP 2020: The Second Shared Task on Cross-Framework and Cross-Lingual Meaning Representation Parsing

Oepen¹,

Abend²,

Abzianidze³

et al. 2020

Proceedings of the CoNLL 2020 Shared Task: Cross-Framework Meaning Representation Parsing

View full text Add to dashboard Cite

The 2020 Shared Task at the Conference for Computational Language Learning (CoNLL) was devoted to Meaning Representation Parsing (MRP) across frameworks and languages. Extending a similar setup from the previous year, five distinct approaches to the representation of sentence meaning in the form of directed graphs were represented in the English training and evaluation data for the task, packaged in a uniform graph abstraction and serialization; for four of these representation frameworks, additional training and evaluation data was provided for one additional language per framework. The task received submissions from eight teams, of which two do not participate in the official ranking because they arrived after the closing deadline or made use of additional training data. All technical information regarding the task, including system submissions, official results, and links to supporting resources and software are available from the task web site at:

show abstract

Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks

Naik

Lehman

Rosé

2022

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

Natural language understanding (NLU) has made massive progress driven by large benchmarks, but benchmarks often leave a long tail of infrequent phenomena underrepresented. We reflect on the question: Have transfer learning methods sufficiently addressed the poor performance of benchmark-trained models on the long tail? We conceptualize the long tail using macro-level dimensions (underrepresented genres, topics, etc.), and perform a qualitative meta-analysis of 100 representative papers on transfer learning research for NLU. Our analysis asks three questions: (i) Which long tail dimensions do transfer learning studies target? (ii) Which properties of adaptation methods help improve performance on the long tail? (iii) Which methodological gaps have greatest negative impact on long tail performance? Our answers highlight major avenues for future research in transfer learning for the long tail. Lastly, using our meta-analysis framework, we perform a case study comparing the performance of various adaptation methods on clinical narratives, which provides interesting insights that may enable us to make progress along these future avenues.

show abstract

Pushing the Limits of AMR Parsing with Self-Learning

Cited by 21 publications

References 19 publications

AMR Parsing with Action-Pointer Transformer

AMR Parsing with Action-Pointer Transformer

MRP 2020: The Second Shared Task on Cross-Framework and Cross-Lingual Meaning Representation Parsing

Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks

Contact Info

Product

Resources

About