Improving Semantic Parsing via Answer Type Inference

Abstract

In this work, we show the possibility of inferring the answer type before solving a factoid question and leveraging the type information to improve semantic parsing. By replacing the topic entity in a question with its type, we are able to generate an abstract form of the question, whose answer corresponds to the answer type of the original question. A bidirectional LSTM model is built to train over the abstract form of questions and infer their answer types. It is also observed that if we convert a question into a statement form, our LSTM model achieves better accuracy. Using the predicted type information to rerank the logical forms returned by AgendaIL, one of the leading semantic parsers, we are able to improve the F1-score from 49.7% to 52.6% on the WE-BQUESTIONS data.

Core techniques of question answering systems over knowledge bases: a survey

“…Moreover differently from QALD many works use the datasets and strategies presented in section 7 to create embeddings (using neural networks) for relations and entities. This is for example the case for Bordes et al 2015, Yang et al 2014, Yavuz et al 2016, STAGG (Yih et al 2015 and Ture & Jojic 2016. The disambiguation problem discussed in section 8 is approached with similar strategies both in QALD and WebQuestions.…”

Recovering Question Answering Errors via Query Revision

“…Ensemble STAGG-RANK (Yavuz et al, 2016) 54.0 QUESREV on STAGG-RANK 54.3 Training Data Preparation. WEBQUESTIONS only provides question-answer pairs along with annotated topic entities.…”

“…However, it is important to note here that Xu et al (2016a) uses DBPedia knowledge base in addition to Freebase and the Wikipedia corpus that we do not utilize. Moreover, applying our approach on the STAGG predictions reranked by (Yavuz et al, 2016), referred as STAGG-RANK in Table 2, leads to a further improvement over a strong ensemble baseline. These suggest that our system captures orthogonal signals to the ones exploited in the base QA models.…”

Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases

“…The former attempts to convert NL questions to logic forms. Recent work focused on approaches based on weak supervision from either external resources (Krishnamurthy and Mitchell, 2012;Berant et al, 2013;Yao and Van Durme, 2014;Hu et al, 2018;Yih et al, 2015;Yavuz et al, 2016), schema matching (Cai and Yates, 2013), or using hand-crafted rules and features (Unger et al, 2012;Berant et al, 2013;Berant and Liang, 2015;Bao et al, 2016;Abujabal et al, 2017;Hu et al, 2018;Bast and Haussmann, 2015;Yih et al, 2015). A thread of research has been explored to generate semantic query graphs from NL questions such as using coarse alignment between phrases and predicates (Berant et al, 2013), searching partial logical forms via an agenda-based strategy (Berant and Liang, 2015), pushing down the disambiguation step into the query evaluation stage (Hu et al, 2018), or exploiting rich syntactic information in NL questions (Xu et al, 2018a,b).…”

“…Macro F 1 SP-based (Berant et al, 2013) 0.357 (Yao and Van Durme, 2014) 0.443 (Wang et al, 2014) 0.453 (Bast and Haussmann, 2015) 0.494 (Berant and Liang, 2015) 0.497 (Yih et al, 2015) 0.525 0.503 (Yavuz et al, 2016) 0.516 (Bao et al, 2016) 0.524 0.471 0.495 (Abujabal et al, 2017) 0.510 (Hu et al, 2018) 0.496 IR-based (Bordes et al, 2014a) 0.392 (Yang et al, 2014) 0.413 (Dong et al, 2015) 0.408 (Bordes et al, 2015) 0.422 (Xu et al, 2016) 0.471 (Hao et al, 2017) 0.429 Our Method: BAMnet w/ gold topic entity 0.557 w/ Freebase Search API 0.497 w/ topic entity predictor 0.518 teresting to note that both (Yih et al, 2015) and (Bao et al, 2016) also use the ClueWeb dataset for learning more accurate semantics. The F1 score of (Yih et al, 2015) drops from 0.525 to 0.509 if ClueWeb information is removed.…”