Few-Shot Text Classification with Pre-Trained Word Embeddings and a Human in the Loop

Abstract

Most of the literature around text classification treats it as a supervised learning problem: given a corpus of labeled documents, train a classifier such that it can accurately predict the classes of unseen documents. In industry, however, it is not uncommon for a business to have entire corpora of documents where few or none have been classified, or where existing classifications have become meaningless. With web content, for example, poor taxonomy management can result in labels being applied indiscriminately, making filtering by these labels unhelpful. Our work aims to make it possible to classify an entire corpus of unlabeled documents using a human-in-the-loop approach, where the content owner manually classifies just one or two documents per category and the rest can be automatically classified. This "few-shot" learning approach requires rich representations of the documents such that those that have been manually labeled can be treated as prototypes, and automatic classification of the rest is a simple case of measuring the distance to prototypes. This approach uses pre-trained word embeddings, where documents are represented using a simple weighted average of constituent word embeddings. We have tested the accuracy of the approach on existing labeled datasets and provide the results here. We have also made code available for reproducing the results we got on the 20 Newsgroups dataset 1 .

Improving the Generalizability of Collaborative Dialogue Analysis With Multi-Feature Embeddings

“…Few-shot learning approach. We benchmark MFeEmb against prior work on conflict prediction, other embedding choices, and FsText, a few-shot model proposed by Bailey and Chopra (2018). Experiments were performed using a 300-dimensional version of MFeEmb where the length of all the three embeddings is the same, i.e., 100.…”

Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks

“…Fewshot text classification entails performing classification after training or tuning a model on only a few examples. Several studies (Yu et al, 2018;Bailey and Chopra, 2018;Geng et al, 2020) have explored various approaches for few-shot text classification, which mainly involve the traditional machine learning techniques for selecting the optimal category sub-samples.…”

An Empirical Study on the Fairness of Pre-trained Word Embeddings

“…Word embeddings are a powerful tool and are applied in variety of Natural Language Processing tasks, such as text classification (Aydogan and Karci, 2020;Alwehaibi and Roy, 2018;Jo and Cinarel, 2019;Bailey and Chopra, 2018;Rescigno et al, 2020) and sentiment analysis (Araque et al, 2017;Rezaeinia et al, 2019;Fu et al, 2017;Ren et al, 2016;Tang et al, 2014). However, analogies such as "Man is to computer programmer as woman is to homemaker" (Bolukbasi et al, 2016a) contain worrisome biases that are present in society and hence embedded in language.…”