Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering

Ma, Kaixin; Francis, Jonathan; Lu, Quanyang; Nyberg, Eric; Oltramari, Alessandro

doi:10.18653/v1/d19-6003

Cited by 57 publications

(64 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In all our implemented models, we use pre-trained LMs as text encoders for s for fair comparison. We do compare our models with those (Ma et al, 2019;Lv et al, 2019;Khashabi et al, 2020) augmented by other text-form external knowledge (e.g., Wikipedia), although we stick to our focus of encoding structured KG. Specifically, we fine-tune BERT-BASE, BERT-LARGE (Devlin et al, 2019), and ROBERTA (Liu et al, 2019b) for multiple-choice questions.…”

Section: Compared Methodsmentioning

confidence: 99%

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering

Feng¹,

Chen²,

Lin³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

132

100

View full text Add to dashboard Cite

Existing work that augment question answering (QA) models with external knowledge (e.g., knowledge graphs) either struggle to model multi-hop relations efficiently, or lack transparency into the model's prediction rationale. In this paper, we propose a novel knowledge-aware approach that equips pretrained language models (PTLMs) with a multi-hop relational reasoning module, named multi-hop graph relation network (MHGRN). It performs multi-hop, multi-relational reasoning over subgraphs extracted from external knowledge graphs. The proposed reasoning module unifies path-based reasoning methods and graph neural networks and results in better interpretability and scalability. We also empirically show its effectiveness and scalability on CommonsenseQA and OpenbookQA datasets, and interpret its behaviors with case studies, with the code for experiments released 1 .

show abstract

Section: Compared Methodsmentioning

confidence: 99%

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering

Feng¹,

Chen²,

Lin³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

132

100

View full text Add to dashboard Cite

show abstract

“…Knowledge Enhanced Methods KagNet (Lin et al, 2019) represents external knowledge as a graph, and then uses graph convolution and LSTM for inference. Ma et al (2019) adopt the BERTbased option comparison network (OCN) for answer prediction, and propose an attention mechanism to perform knowledge integration using relevant triples. Lv et al (2020) propose a GNN-based inference model on conceptual network relationships and heterogeneous graphs of Wikipedia sentences.…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we describe the method to extract knowledge facts from knowledge graph in details. Once the knowledge is determined, we can choose the appropriate integration mechanism for further knowledge injection, such as attention mechanism (Sun et al, 2018;Ma et al, 2019), pre-training tasks (He et al, 2019) and multi-task training . Given a question Q and a candidate answer O, we first identify the entity and its type in the text by entity linking.…”

Section: Knowledge Acquisitionmentioning

confidence: 99%

Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text

Li¹,

Hu²,

Chen³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Machine reading comprehension (MRC) has achieved significant progress on the open domain in recent years, mainly due to large-scale pre-trained language models. However, it performs much worse in specific domains such as the medical field due to the lack of extensive training data and professional structural knowledge neglect. As an effort, we first collect a large scale medical multi-choice question dataset (more than 21k instances) for the National Licensed Pharmacist Examination in China. It is a challenging medical examination with a passing rate of less than 14.2% in 2018. Then we propose a novel reading comprehension model KMQA, which can fully exploit the structural medical knowledge (i.e., medical knowledge graph) and the reference medical plain text (i.e., text snippets retrieved from reference books). The experimental results indicate that the KMQA outperforms existing competitive models with a large margin and passes the exam with 61.8% accuracy rate on the test set.

show abstract

“…Our proposed model solves these problems Group 2 comprises models that use additional external knowledge while using RoBERTa as the encoding layer, similar to the proposed model. KagNet [22], and HyKAS 2.0 [23] use ConceptNet as the external knowledge source, similar to the proposed model. RoBERTa + KE uses wiki docs as the external knowledge source to search for sentences while RoBERTa + IR uses the Open Mind Common Sense (OMCS) [24] and searches for sentences using a search engine.…”

Section: Model Performance For Commonsenseqamentioning

confidence: 99%