Neural Duplicate Question Detection without Labeled Training Data

Abstract

Supervised training of neural models to duplicate question detection in community Question Answering (cQA) requires large amounts of labeled question pairs, which are costly to obtain. To minimize this cost, recent works thus often used alternative methods, e.g., adversarial domain adaptation. In this work, we propose two novel methods: (1) the automatic generation of duplicate questions, and (2) weak supervision using the title and body of a question. We show that both can achieve improved performances even though they do not require any labeled data. We provide comprehensive comparisons of popular training strategies, which provides important insights on how to 'best' train models in different scenarios. We show that our proposed approaches are more effective in many cases because they can utilize larger amounts of unlabeled data from cQA forums. Finally, we also show that our proposed approach for weak supervision with question title and body information is also an effective method to train cQA answer selection models without direct answer supervision.

MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale

“…We optimize the binary cross-entropy loss. Similar techniques achieve state-of-the-art results on many related datasets (Garg et al, 2020;Mass et al, 2019;Rücklé et al, 2019b).…”

“…Poerner and Schütze (2019) adapt the combination of different sentence embeddings to individual target domains. Rücklé et al (2019b) use weakly supervised training, self-supervised training methods, and question generation. Similar approaches were also explored in ad-hoc retrieval (Zhang et al, 2020;Ma et al, 2020;MacAvaney et al, 2019).…”

“…We use data from each of these 140 English forums and train domain-specific models for semantic text matching. This has recently become feasible with self-supervised training methods such as WS-TB (Rücklé et al, 2019b), in which the question title is considered as a query to retrieve the question body (the detailed description of the question). This requires no labeled training instances and thus allows us to scale our experiments to 140 source domains which we can transfer from.…”

“…Reasons are that (1) there exist a large number of domains, and (2) in-domain training data is often scarce. Previous work proposed domain adaptation techniques (Poerner and Schütze, 2019;Shah et al, 2018), training with unlabeled data (Rücklé et al, 2019b), and shallow architectures (Rücklé et al, 2019a). However, these approaches result in entirely separate models that are specialized to individual target domains.…”

“…By leveraging self-supervised training signals of ques-tion title-body pairs, we analyze a large number of models specialized on diverse domains. We utilize the training method provided by Rücklé et al (2019b) and train adapter modules (Rebuffi et al, 2017;Houlsby et al, 2019) within BERT (Devlin et al, 2019) for each of the 140 English Stack-Exchange forums. Adapters considerably reduce storage requirements by training only a small number of additional parameters while keeping the pre-trained BERT weights fixed.…”