Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches

Storks, Shane; Gao, Qiaozi; Chai, Joyce

doi:10.48550/arxiv.1904.01172

Cited by 33 publications

(40 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They would only affect the model performance slightly. In contrast, complex intermediate tasks are generally rather beneficial to promote the model, e.g., natural language inference [18], and question answering [19]. Thus, they expect the model to have strong capabilities such as perceiving interrelations.…”

Section: B Transfer Learning and Adaptive Schemesmentioning

confidence: 99%

Assemble Foundation Models for Automatic Code Summarization

Gu¹,

Salza²,

Gall³

2022

Preprint

View full text Add to dashboard Cite

Automatic code summarization is beneficial to software development and maintenance since it reduces the burden of manual tasks. Currently, artificial intelligence is undergoing a paradigm shift. The foundation models pretrained on massive data and finetuned to downstream tasks surpass specially customized models. This trend inspired us to consider reusing foundation models instead of learning from scratch. Based on this, we propose a flexible and robust approach for automatic code summarization based on neural networks. We assemble available foundation models, such as CodeBERT and GPT-2, into a single model named AdaMo. Moreover, we utilize Gaussian noise as the simulation of contextual information to optimize the latent representation. Furthermore, we introduce two adaptive schemes from the perspective of knowledge transfer, namely continuous pretraining and intermediate finetuning, and design intermediate stage tasks for general sequence-to-sequence learning. Finally, we evaluate AdaMo against a benchmark dataset for code summarization, by comparing it with state-of-the-art models.

show abstract

Section: B Transfer Learning and Adaptive Schemesmentioning

confidence: 99%

Assemble Foundation Models for Automatic Code Summarization

Gu¹,

Salza²,

Gall³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The growing volume of the published biomedical literature, such as clinical reports 1 and health literacy 2 demands more precise and generalized biomedical natural language processing (BioNLP) tools for information extraction. The recent advancement of using deep learning (DL) in natural language processing (NLP) has fueled the advancements in the development of pre-trained language models (LMs) that can be applied to a range of tasks in the BioNLP domains 3 .…”

Section: Background and Summarymentioning

confidence: 99%

Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERT

Naseem¹,

Dunn²,

Khushi³

et al. 2021

Preprint

View full text Add to dashboard Cite

The availability of biomedical text data and advances in natural language processing (NLP) have made new applications in biomedical NLP possible. Language models trained or fine-tuned using domain-specific corpora can outperform general models, but work to date in biomedical NLP has been limited in terms of corpora and tasks. We present BioALBERT, a domain-specific adaptation of A Lite Bidirectional Encoder Representations from Transformers (ALBERT), trained on biomedical (PubMed and PubMed Central) and clinical (MIMIC-III) corpora and fine-tuned for 6 different tasks across 20 benchmark datasets. Experiments show that BioALBERT outperforms the state-of-the-art on named-entity recognition (+11.09% BLURB score improvement), relation extraction (+0.80% BLURB score), sentence similarity (+1.05% BLURB score), document classification (+0.62% F1-score), and question answering (+2.83% BLURB score). It represents a new state-of-the-art in 17 out of 20 benchmark datasets. By making BioALBERT models and data available, our aim is to help the biomedical NLP community avoid computational costs of training and establish a new set of baselines for future efforts across a broad range of biomedical NLP tasks.

show abstract

“…While KG-augmented models are applicable to any KG-related CSR task (e.g., natural language inference), we consider multi-choice QA, which is one of the most popular CSR tasks (Sap et al, 2020;Storks et al, 2019). Given a question q, set of answer choices A = {a i }, and target answer a * ∈ A, the QA model's objective is to predict a confidence probability p(q, a i ) for each (q, a i ) pair, so that a * = arg max a i ∈A p(q, a i ).…”

Section: Commonsense Qamentioning

confidence: 99%

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Chan

Xu²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. Although some works have attempted to explain the behavior of such KG-augmented models by indicating which KG inputs are salient (i.e., important for the model's prediction), it is not always clear how these explanations should be used to make the model better. In this paper, we explore whether KG explanations can be used as supervision for teaching these KG-augmented models how to filter out unhelpful KG information. To this end, we propose SALKG, a simple framework for learning from KG explanations of both coarse (Is the KG salient?) and fine (Which parts of the KG are salient?) granularity. Given the explanations generated from a task's training set, SALKG trains KG-augmented models to solve the task by focusing on KG information highlighted by the explanations as salient. Across two popular commonsense QA benchmarks and three KG-augmented models, we find that SALKG's training process can consistently improve model performance. 1 * Equal contribution. † Work done while TG interned remotely at USC. 1 Code will be released at github.com/INK-USC/SalKG.

show abstract

Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches

Cited by 33 publications

References 0 publications

Assemble Foundation Models for Automatic Code Summarization

Assemble Foundation Models for Automatic Code Summarization

Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERT

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Contact Info

Product

Resources

About