Curriculum Learning for Natural Language Understanding

Xu, Benfeng; Zhang, Licheng; Mao, Zhendong; Wang, Quan; Xie, Hongtao; Zhang, Yongdong

doi:10.18653/v1/2020.acl-main.542

Cited by 112 publications

(103 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Curriculum Learning is a learning strategy firstly proposed by Bengio et al (2009) that trains a neural network better through increasing data complexity of training data. It is broadly adopted in many NLP domains (Platanios et al, 2019;Huang and Du, 2019;Xu et al, 2020). In this work, since data with rich related arguments is easier to be learned than those without extra inputs, we promote the training of our student model by gradually increasing the learning complexity of the distillation process by decreasing the proportion of given arguments.…”

Section: Related Workmentioning

confidence: 99%

Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction

Wei¹,

Sun²,

Zhang³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Implicit Event Argument Extraction seeks to identify arguments that play direct or implicit roles in a given event. However, most prior works focus on capturing direct relations between arguments and the event trigger. The lack of reasoning ability brings many challenges to the extraction of implicit arguments. In this work, we present a Frame-aware Event Argument Extraction (FEAE) learning framework to tackle this issue through reasoning in event frame-level scope. The proposed method leverages related arguments of the expected one as clues to guide the reasoning process. To bridge the gap between oracle knowledge used in the training phase and the imperfect related arguments in the test stage, we further introduce a curriculum knowledge distillation strategy to drive a final model that could operate without extra inputs through mimicking the behavior of a well-informed teacher model. Experimental results demonstrate FEAE obtains new state-ofthe-art performance on the RAMS dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction

Wei¹,

Sun²,

Zhang³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…The above implementations lack thinking about the learning process. The process of human learning often goes from easy to difficult (Xu et al, 2020). Especially for the correlated tasks, humans can dig into the hidden knowledge and extract them from the easy tasks for completing the hard ones.…”

Section: Progressive Tasksmentioning

confidence: 99%

An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization

Zhou¹,

Cai²,

Zhang³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Medical named entity recognition (NER) and normalization (NEN) are fundamental for constructing knowledge graphs and building QA systems. Existing implementations for medical NER and NEN are suffered from the error propagation between the two tasks. The mispredicted mentions from NER will directly influence the results of NEN. Therefore, the NER module is the bottleneck of the whole system. Besides, the learnable features for both tasks are beneficial to improving the model performance. To avoid the disadvantages of existing models and exploit the generalized representation across the two tasks, we design an end-to-end progressive multi-task learning model for jointly modeling medical NER and NEN in an effective way. There are three level tasks with progressive difficulty in the framework. The progressive tasks can reduce the error propagation with the incremental task settings which implies the lower level tasks gain the supervised signals other than errors from the higher level tasks to improve their performances. Besides, the context features are exploited to enrich the semantic information of entity mentions extracted by NER. The performance of NEN profits from the enhanced entity mention features. The standard entities from knowledge bases are introduced into the NER module for extracting corresponding entity mentions correctly. The empirical results on two publicly available medical literature datasets demonstrate the superiority of our method over nine typical methods.

show abstract

“…For Subtask 2, they include several tokens and embeddings based on document structure into input representation for BART. Instead of random order of the training instances, they propose to apply curriculum learning (Xu et al, 2020) based on the computed task difficulty level for each task respectively. The final submission on Subtask 2 is based on the span prediction by a single model.…”

Section: Ku Nlpmentioning

confidence: 99%

DialDoc 2021 Shared Task: Goal-Oriented Document-grounded Dialogue Modeling

Feng¹

2021

Proceedings of the 1st Workshop on Document-Grounded Dialogue and Conversational Question Answering (DialDoc 2021)

View full text Add to dashboard Cite

We present the results of Shared Task at Workshop DialDoc 2021 that is focused on document-grounded dialogue and conversational question answering. The primary goal of this Shared Task is to build goal-oriented information-seeking conversation systems that can identify the most relevant knowledge in the associated document for generating agent responses in natural language. It includes two subtasks on predicting agent responses: the first subtask is to predict the grounding text span in the given document for next agent response; the second subtask is to generate agent response in natural language given the context. Many submissions outperform baseline significantly. For the first task, the best-performing system achieved 67.1 Exact Match and 76.3 F1. For the second subtask, the best system achieved 41.1 SacreBLEU and highest rank by human evaluation.

show abstract

Curriculum Learning for Natural Language Understanding

Cited by 112 publications

References 25 publications

Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction

Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction

An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization

DialDoc 2021 Shared Task: Goal-Oriented Document-grounded Dialogue Modeling

Contact Info

Product

Resources

About