ALICE: Active Learning with Contrastive Natural Language Explanations

Liang, Weixin; Zou, James; Yu, Zhou

doi:10.18653/v1/2020.emnlp-main.355

Cited by 24 publications

(22 citation statements)

References 36 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Minimal pairs have also been used to design controlled experiments and probe neural models' ability to capture various linguistic phenomena (Gulordava et al, 2018;Ettinger et al, 2018;Futrell et al, 2019;Gardner et al, 2020;Schuster et al, 2020). Finally, Liang et al (2020) use contrastive explanations as part of an active learning framework to improve data efficiency. Our work shares the objective of Liang et al (2020) to improve data efficiency, but is methodologically closer to probing work that uses minimal pairs to represent specific linguistic features.…”

Section: Discovering and Detecting Dialect Featuresmentioning

confidence: 99%

“…Finally, Liang et al (2020) use contrastive explanations as part of an active learning framework to improve data efficiency. Our work shares the objective of Liang et al (2020) to improve data efficiency, but is methodologically closer to probing work that uses minimal pairs to represent specific linguistic features.…”

Section: Discovering and Detecting Dialect Featuresmentioning

confidence: 99%

See 1 more Smart Citation

Learning to Recognize Dialect Features

Demszky

Sharma

Clark

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Building NLP systems that serve everyone requires accounting for dialect differences. But dialects are not monolithic entities: rather, distinctions between and within dialects are captured by the presence, absence, and frequency of dozens of dialect features in speech and text, such as the deletion of the copula in "He ∅ running". In this paper, we introduce the task of dialect feature detection, and present two multitask learning approaches, both based on pretrained transformers. For most dialects, largescale annotated corpora for these features are unavailable, making it difficult to train recognizers. We train our models on a small number of minimal pairs, building on how linguists typically define dialect features. Evaluation on a test set of 22 dialect features of Indian English demonstrates that these models learn to recognize many features with high accuracy, and that a few minimal pairs can be as effective for training as thousands of labeled examples. We also demonstrate the downstream applicability of dialect feature detection both as a measure of dialect density and as a dialect classifier.

show abstract

Section: Discovering and Detecting Dialect Featuresmentioning

confidence: 99%

Section: Discovering and Detecting Dialect Featuresmentioning

confidence: 99%

Learning to Recognize Dialect Features

Demszky

Sharma

Clark

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…For examples, there can be color attributes like "white", size attributes like "large", and action attributes like "standing". Attributes are important sources of information beyond the coarse-grained object classes (Liang et al, 2020c). Each edge in the scene graph denotes relation between two connected objects.…”

Section: C2 Additional Dataset Informationmentioning

confidence: 99%

GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering

Liang¹,

Jiang²,

Liu³

2021

Proceedings of the Third Workshop on Multimodal Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Images are more than a collection of objects or attributes -they represent a web of relationships among interconnected objects. Scene Graph has emerged as a new modality as a structured graphical representation of images. Scene Graph encodes objects as nodes connected via pairwise relations as edges. To support question answering on scene graphs, we propose GraphVQA, a language-guided graph neural network framework that translates and executes a natural language question as multiple iterations of message passing among graph nodes. We explore the design space of GraphVQA framework, and discuss the trade-off of different design choices. Our experiments on GQA dataset show that GraphVQA outperforms the state-of-the-art model by a large margin (88.43% vs. 94.78%).

show abstract

“…Moreover, most pool-based AL strategies query class information of instances. However, recently, Liang et al [158] proposed the strategy active learning with contrastive natural language explanations (ALICE). It uses queries of the form "How would you differentiate between the class y ∈ Ω Y and class y ∈ Ω Y ?"…”

Section: A Active Learning For Classificationmentioning

confidence: 99%

A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification

et al. 2021

View full text Add to dashboard Cite

Pool-based active learning (AL) aims to optimize the annotation process (i.e., labeling) as the acquisition of annotations is often time-consuming and therefore expensive. For this purpose, an AL strategy queries annotations intelligently from annotators to train a high-performance classification model at a low annotation cost. Traditional AL strategies operate in an idealized framework. They assume a single, omniscient annotator who never gets tired and charges uniformly regardless of query difficulty. However, in real-world applications, we often face human annotators, e.g., crowd or in-house workers, who make annotation mistakes and can be reluctant to respond if tired or faced with complex queries. Recently, a wide range of novel AL strategies has been proposed to address these issues. They differ in at least one of the following three central aspects from traditional AL: (1) They explicitly consider (multiple) human annotators whose performances can be affected by various factors, such as missing expertise. (2) They generalize the interaction with human annotators by considering different query and annotation types, such as asking an annotator for feedback on an inferred classification rule. (3) They take more complex cost schemes regarding annotations and misclassifications into account. This survey provides an overview of these AL strategies and refers to them as real-world AL. Therefore, we introduce a general real-world AL strategy as part of a learning cycle and use its elements, e.g., the query and annotator selection algorithm, to categorize about 60 real-world AL strategies. Finally, we outline possible directions for future research in the field of AL.

show abstract

ALICE: Active Learning with Contrastive Natural Language Explanations

Cited by 24 publications

References 36 publications

Learning to Recognize Dialect Features

Learning to Recognize Dialect Features

GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering

A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification

Contact Info

Product

Resources

About