MilkQA: A Dataset of Consumer Questions for the Task of Answer Selection

Abstract

Abstract-We introduce MilkQA, a question answering dataset from the dairy domain dedicated to the study of consumer questions. The dataset contains 2,657 pairs of questions and answers, written in the Portuguese language and originally collected by the Brazilian Agricultural Research Corporation (Embrapa). All questions were motivated by real situations and written by thousands of authors with very different backgrounds and levels of literacy, while answers were elaborated by specialists from Embrapa's customer service. Our dataset was filtered and anonymized by three human annotators. Consumer questions are a challenging kind of question that is usually employed as a form of seeking information. Although several question answering datasets are available, most of such resources are not suitable for research on answer selection models for consumer questions. We aim to fill this gap by making MilkQA publicly available. We study the behavior of four answer selection models on MilkQA: two baseline models and two convolutional neural network archictetures. Our results show that MilkQA poses real challenges to computational models, particularly due to linguistic characteristics of its questions and to their unusually longer lengths. Only one of the experimented models gives reasonable results, at the cost of high computational requirements.

PerAnSel: A Novel Deep Neural Network-Based System for Persian Question Answering

“…The MilkQA dataset [ 57 ] is a closed-domain dataset prepared for the Portuguese language. The questions are about dairy.…”

Pirá: A Bilingual Portuguese-English Dataset for Question-Answering about the Ocean

“…We mention, for instance, versions of SQuAD 3 and GLUE 4 in Portuguese. The two exceptions are the ENEM-Challeng [36] and MilkQA [7]. The ENEM-Challenge is based on ENEM, the entrance examination valid for almost all universities in Brazil.…”

“…In our dataset, however, the rewriting process was executed by a different individual, bringing an additional factor of diversity to the generation of QA sets. 7 Some examples of paraphrases are shown in Table 4.…”

AIA-BDE: um Corpo de Perguntas, Variações e outras Anotações

“…No que concerne especificamente ao português, e tanto quanto conhecemos, o mais próximo com uma coleção de FAQ e respetivas respostas será a coleção MilkQA (Criscuolo et al, 2017), que inclui perguntas colocadas de forma mais densa, no domínio dos laticínios, seguidas pelas suas respostas. A disponibilização do corpo AIA-BDE é mais uma contribuição neste sentido, já que inclui FAQ em português, no domínio da administração pública, e está preparada para avaliar um conjunto de tarefas relevantes para sistemas de IR, RAP, e mesmo diálogo, com foco na associação de interações em linguagem natural com perguntas conhecidas.…”