Neural Network Acceptability Judgments

Warstadt, Alex; Singh, Amanpreet; Bowman, Samuel R.

doi:10.1162/tacl_a_00290

Cited by 614 publications

(446 citation statements)

References 48 publications

Supporting

Mentioning

371

Contrasting

Unclassified

Order By: Relevance

“…nine natural language understanding (NLU) tasks. As shown in Table 1, it includes question answering (Rajpurkar et al, 2016), linguistic acceptability (Warstadt et al, 2018), sentiment analysis (Socher et al, 2013), text similarity (Cer et al, 2017), paraphrase detection (Dolan and Brockett, 2005), and natural language inference (NLI) Bar-Haim et al, 2006;Giampiccolo et al, 2007;Bentivogli et al, 2009;Levesque et al, 2012;Williams et al, 2018). The diversity of the tasks makes GLUE very suitable for evaluating the generalization and robustness of NLU models.…”

Section: Modelmentioning

confidence: 99%

Multi-Task Deep Neural Networks for Natural Language Understanding

Liu¹,

He²,

Chen³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

896

713

View full text Add to dashboard Cite

We present MT-DNN 1 , an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models. Built upon PyTorch and Transformers, MT-DNN is designed to facilitate rapid customization for a broad spectrum of NLU tasks, using a variety of objectives (classification, regression, structured prediction) and text encoders (e.g., RNNs, BERT, RoBERTa, UniLM). A unique feature of MT-DNN is its built-in support for robust and transferable learning using the adversarial multi-task learning paradigm. To enable efficient production deployment, MT-DNN supports multitask knowledge distillation, which can substantially compress a deep neural model without significant performance drop. We demonstrate the effectiveness of MT-DNN on a wide range of NLU applications across general and biomedical domains. The software and pretrained models will be publicly available at https://github.com/namisan/mt-dnn.

show abstract

Section: Modelmentioning

confidence: 99%

Multi-Task Deep Neural Networks for Natural Language Understanding

Liu¹,

He²,

Chen³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

896

713

View full text Add to dashboard Cite

show abstract

“…The General Language Understanding Evaluation (GLUE) benchmark ) is a collection of diverse natural language understanding tasks (Warstadt et al, 2018;Socher et al, 2013;Dolan and Brockett, 2005;Agirre et al, 2007;Williams et al, 2018;Rajpurkar et al, 2016;Dagan et al, 2006;Levesque et al, 2011), which is the main benchmark used in Devlin et al (2019).…”

Section: Gluementioning

confidence: 99%

ERNIE: Enhanced Language Representation with Informative Entities

Zhang¹,

Han²,

Liu³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

1,012

643

View full text Add to dashboard Cite

Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks. However, the existing pre-trained language models rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better language understanding. We argue that informative entities in KGs can enhance language representation with external knowledge. In this paper, we utilize both large-scale textual corpora and KGs to train an enhanced language representation model (ERNIE), which can take full advantage of lexical, syntactic, and knowledge information simultaneously. The experimental results have demonstrated that ERNIE achieves significant improvements on various knowledge-driven tasks, and meanwhile is comparable with the state-of-the-art model BERT on other common NLP tasks. The source code and experiment details of this paper can be obtained from https:// github.com/thunlp/ERNIE. * indicates equal contribution † Corresponding author: Z.Liu(liuzy@tsinghua.edu.cn) is_a is_a Song Book a u th o r c o m p o s e r Bob Dylan Chronicles: Volume One Blowin' in the wind Songwriter Writer is_a is_a Bob Dylan wrote

show abstract

“…The Corpus of Linguistic Acceptability (CoLA) is a binary classification task. The goal of this task is to predict whether an English sentence is linguistically acceptable or not (Warstadt et al, 2018). Table 8 presents the accuracy scores of BERT and DISP on the CoLA dataset with one adversarial attack of each type.…”

Section: Resultsmentioning

confidence: 99%

Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification

Zhou¹,

Jiang²,

Chang³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Adversarial attacks against machine learning models have threatened various real-world applications such as spam filtering and sentiment analysis. In this paper, we propose a novel framework, learning to discriminate perturbations (DISP), to identify and adjust malicious perturbations, thereby blocking adversarial attacks for text classification models. To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations. For each potential perturbation, an embedding estimator learns to restore the embedding of the original word based on the context and a replacement token is chosen based on approximate kNN search. DISP can block adversarial attacks for any NLP model without modifying the model structure or training procedure. Extensive experiments on two benchmark datasets demonstrate that DISP significantly outperforms baseline methods in blocking adversarial attacks for text classification. In addition, in-depth analysis shows the robustness of DISP across different situations.

show abstract

Neural Network Acceptability Judgments

Cited by 614 publications

References 48 publications

Multi-Task Deep Neural Networks for Natural Language Understanding

Multi-Task Deep Neural Networks for Natural Language Understanding

ERNIE: Enhanced Language Representation with Informative Entities

Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification

Contact Info

Product

Resources

About