COALA: A Neural Coverage-Based Approach for Long Answer Selection with Small Data

Abstract

Current neural network based community question answering (cQA) systems fall short of (1) properly handling long answers which are common in cQA; (2) performing under small data conditions, where a large amount of training data is unavailable—i.e., for some domains in English and even more so for a huge number of datasets in other languages; and (3) benefiting from syntactic information in the model—e.g., to differentiate between identical lexemes with different syntactic roles. In this paper, we propose COALA, an answer selection approach that (a) selects appropriate long answers due to an effective comparison of all question-answer aspects, (b) has the ability to generalize from a small number of training examples, and (c) makes use of the information about syntactic roles of words. We show that our approach outperforms existing answer selection models by a large margin on six cQA datasets from different domains. Furthermore, we report the best results on the passage retrieval benchmark WikiPassageQA.

Neural Duplicate Question Detection without Labeled Training Data

“…To test whether our strategy to train models with title-body pairs is also suitable for answer selection, we use the data and code of Rücklé et al (2019a) and train two different types of models with WS-TB on their five datasets that are based on StackExchange Apple, Aviation, Academia, Cooking, and Travel. We train (1) a siamese BiLSTM, which learns question and answer representations; and (2) their neural relevance matching model COALA.…”

“…https://github.com/huggingface/ pytorch-transformers same as for BiLSTM (e.g., loss calculation). We train the models until they do not improve for at least 20 epochs, and we restore the weights of the epoch that obtained the best development score.For all other datasets (AskUbuntu-Lei and Answer Selection datasets) we add BERT to the experimental software ofRücklé et al (2019a). We do not include it in the software ofLei et al (2016) because it is tightly coupled to the Theano framework, which is not actively maintained.…”

MICRON: Multigranular Interaction for Contextualizing RepresentatiON in Non-factoid Question Answering

“…Output matrix η nm ∈ R |Q |× |P | contains the relevance scores of all pairs between n-grams in query and m-grams in answer. From η nm , we conduct a row-wise max-pooling to obtain A nm , relaxing the length constraint in interactions (Rücklé et al, 2019).…”

“…InsuranceQA WikiPassageQA Accuracy MAP MRR P@5 P@10 nDCG R@5 R@10 R@20 IR Based (Rücklé et al, 2019) . COALA syntax-aware (Rücklé and Gurevych, 2017) is a variant of COALA using dependency parse trees (Schuster and Manning, 2016).…”

“…Later work relaxes 1:1 constraint, to 1:N and M:N, by allowing a match to n-gram (1:N) or a match between query and answer bi-grams (2:2). A state-of-the-art in this category (Rücklé et al, 2019), shown in Figure 1b, uses bi-gram Convolutional Neural Network (CNN) to represent query/answer bi-grams and their interactions. Similar architecture was generalized for N:N or N:M matches (Song et al, 2019;Chen et al, 2018), which may introduce a new challenge of multigranular interaction we discuss later.…”