An Interface for Annotating Science Questions

Boratko, Michael; Padigela, Harshit; Mikkilineni, Divyendra; Yuvraj, Pritish; Das, Rajarshi; McCallum, Andrew; Chang, Maria Hsia; Fokoue, Achille; Kapanipathi, Pavan; Mattei, Nicholas; Musa, Ryan; Talamadupula, Kartik; Witbrock, Michael

doi:10.18653/v1/d18-2018

Cited by 4 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It contains a dataset with 2,590 multiple-choice questions written for the primary school science exam (Clark et al, 2018). In this competition, Boratko et al (2018aBoratko et al ( , 2018b verified the rewritten query effect on the pretrained DrQA model (Chen et al, 2017), and the result proved that the score increased by 0.42, thus quantitatively verifying the validity of the query rewriting. Musa et al (2018) is based on the Seq2seq model and the NCRF model, supplemented by word vectors pretrained on the knowledge graph as a priori knowledge to generate multiple new queries by identifying key items from the OQ.…”

Section: Related Work 21 Query Rewritingmentioning

confidence: 71%

A novel word-graph-based query rewriting method for question answering

Yan

Dang

et al. 2023

DTA

View full text Add to dashboard Cite

PurposeQuestion answering (QA) answers the questions asked by people in the form of natural language. In the QA, due to the subjectivity of users, the questions they query have different expressions, which increases the difficulty of text retrieval. Therefore, the purpose of this paper is to explore new query rewriting method for QA that integrates multiple related questions (RQs) to form an optimal question. Moreover, it is important to generate a new dataset of the original query (OQ) with multiple RQs.Design/methodology/approachThis study collects a new dataset SQuAD_extend by crawling the QA community and uses word-graph to model the collected OQs. Next, Beam search finds the best path to get the best question. To deeply represent the features of the question, pretrained model BERT is used to model sentences.FindingsThe experimental results show three outstanding findings. (1) The quality of the answers is better after adding the RQs of the OQs. (2) The word-graph that is used to model the problem and choose the optimal path is conducive to finding the best question. (3) Finally, BERT can deeply characterize the semantics of the exact problem.Originality/valueThe proposed method can use word-graph to construct multiple questions and select the optimal path for rewriting the question, and the quality of answers is better than the baseline. In practice, the research results can help guide users to clarify their query intentions and finally achieve the best answer.

show abstract

Section: Related Work 21 Query Rewritingmentioning

confidence: 71%

A novel word-graph-based query rewriting method for question answering

Yan

Dang

et al. 2023

DTA

View full text Add to dashboard Cite

show abstract

“…For each question, we provided English translations as not all annotators were native speakers of the questions' language. We followed the procedure and re-used the annotation types presented in earlier work Boratko et al, 2018). However, as they were designed mainly for Nature Science questions, we extended them with two new annotation types: "Domain Facts and Knowledge" and "Negation" (see Appendix C for examples).…”

Section: Reasoning and Knowledge Typesmentioning

confidence: 99%

“…For our reasoning and knowledge type annotations, we followed the procedure and re-used the annotation types presented in Boratko et al, 2018). However, as they were designed mainly for Natural Science questions, we had to extend them with two new types:…”

Section: Reasoning and Knowledge Typesmentioning

confidence: 99%

EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering

Hardalov¹,

Mihaylov²,

Zlatkova³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose Eχαµs -a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations. We collected more than 24,000 highquality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others.Eχαµs offers a fine-grained evaluation framework across multiple languages and subjects, which allows precise analysis and comparison of various models. We perform various experiments with existing top-performing multilingual pre-trained models and we show that Eχαµs offers multiple challenges that require multilingual knowledge and reasoning in multiple domains. We hope that Eχαµs will enable researchers to explore challenging reasoning and knowledge transfer methods and pretrained models for school question answering in various languages which was not possible before. The data, code, pre-trained models, and evaluation are available at http:// github.com/mhardalov/exams-qa.

show abstract

“…In contrast, domain-agnostic AQC is applied in information query or dialogue interactions in which the class labels may comprise question types (e.g., true/false, procedural) [7] or reasoning capabilities (e.g., multi-hop, comparison, algebraic) [8,9]. To enhance the effectiveness of deliberate practice [10], assessment questions are classified into their re-Chapter 1: Introduction spective cognitive complexities (e.g., synthesis, evaluation) for instructors to determine learners' proficiencies [11][12][13][14].…”

Section: Motivationmentioning

confidence: 99%

“…Questions have also been labeled according to reasoning abilities. The ARC dataset of the AI2 Reasoning Challenge has been annotated by subject-matter experts according to several knowledge and reasoning types [8,9]. Due to overlapping categories, questions belonging to three mutually exclusive class labels (Basic facts, Linguistic matching, Hypothetical ) have been selected.…”

Section: Topic Regularization Mechanismmentioning

confidence: 99%

Domain-agnostic document and question classification using natural language processing techniques

Supraja¹

View full text Add to dashboard Cite

I would like to dedicate this thesis to two special angels: God and my dear parents, without whom I could not have completed this challenging, yet fruitful PhD journey. I have no words to express my gratitude for every single thing my parents have done for my education and for my whole life. I cannot do anything to repay their love, affection, care, and concern for me.I would like to extend my sincere thanks to my husband, all my family members and friends who have supported me in one way or another. First and foremost, I would like to sincerely express my gratitude to my advisor A/Prof Andy W. H. Khong for his continuous support and guidance throughout my Ph.D. journey. His constant motivation and encouragement has transformed me tremendously.His advice has always been a great tonic to boost my morale and confidence level.

show abstract

An Interface for Annotating Science Questions

Cited by 4 publications

References 10 publications

A novel word-graph-based query rewriting method for question answering

A novel word-graph-based query rewriting method for question answering

EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering

Domain-agnostic document and question classification using natural language processing techniques

Contact Info

Product

Resources

About