Learning from noisy out-of-domain corpus using dataless classification

Abstract

In real-world applications, text classification models often suffer from a lack of accurately labelled documents. The available labelled documents may also be out of domain, making the trained model not able to perform well in the target domain. In this work, we mitigate the data problem of text classification using a two-stage approach. First, we mine representative keywords from a noisy out-of-domain data set using statistical methods. We then apply a dataless classification method to learn from the automatically selected keywords and unlabelled in-domain data. The proposed approach outperformed various supervised learning and dataless classification baselines by a large margin. We evaluated different keyword selection methods intrinsically and extrinsically by measuring their impact on the dataless classification accuracy. Last but not least, we conducted an in-depth analysis of the behaviour of the classifier and explained why the proposed dataless classification method outperformed supervised learning counterparts.

Bootstrapping Large-Scale Fine-Grained Contextual Advertising Classifier from Wikipedia

“…use. Jin et al (2020) demonstrated that the choice of seed keywords has a significant impact on the model's accuracy. STM,S label is the result of STM using only unigrams in the category name as seed keywords.…”

“…To investigate the contribution of the in-domain unlabeled document to STM's superior performance, we trained an STM model with the manually-curated keywords in Jin et al (2020) and the Wikipedia dataset we used to train wiki2cat (denoted as STM,D wiki ). There is a noticeable decrease in performance in STM,D wiki without indomain unlabeled documents.…”

“…Both Doc2vec and STM require unlabeled training corpus. We copy the coarsegrained classification result for Doc2vec, STM, and STM,S label from Jin et al (2020). For fine-grained classification, we train Doc2vec and STM,S label using the same set of Wikipedia documents as in wiki2cat.…”

Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation

“…We use either the category name or trivial keywords (e.g., "good" and "bad" for sentiment classification tasks) as the only input seed word and use a keyword expansion algorithm to mine more candidate keywords. We apply pmi-f req (Equation 1) following Jin et al (2020). It is a product of the logarithm of the candidate keyword w's document frequency and the point-wise mutual information between w and the seed word s. The higher the pmi-f req score, the more strongly the candidate keyword is associated with the seed word s. Additionally, we filter the mined keywords based on their part-of-speech tag depending on the classification task.…”

“…Weakly-supervised text classification eliminates the need for any labeled document and induces classifiers with only a handful of carefully chosen seed words. However, some researchers pointed out that the choice of seed words has a significant impact on the performance of weakly-supervised models (Li et al, 2018;Jin et al, 2020). The vast majority of previous work assumed high-quality seed words are given.…”