Incorporating External Knowledge to Enhance Tabular Reasoning

Neeraja, J.; Gupta, Vivek; Srikumar, Vivek

doi:10.18653/v1/2021.naacl-main.224

Cited by 21 publications

(31 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pre-training over semi-structured data Past work on pre-training over tables focused on reasoning over tables and knowledge-bases Yin et al, 2020;Herzig et al, 2020;Müller et al, 2021;Yu et al, 2021;Neeraja et al, 2021b) We evaluate our model, PReasM, on three reasoning-focused RC datasets and show that it leads to substantial improvements in all cases. Moreover, we thoroughly analyze the performance of PReasM and show that our approach dramatically improves performance on questions that require reasoning skills that were not acquired during the original pre-training, while maintaining comparable performance on other question types.…”

Section: Related Workmentioning

confidence: 97%

“…We use tables from English Wikipedia 1 to generate D syn . English Wikipedia includes millions of tables with high lexical and domain diversity (Fetahu et al, 2019;Chen et al, 2020b;Gupta et al, 2020b;Talmor et al, 2021;Nan et al, 2021;Neeraja et al, 2021a). We first extract from Wikipedia all tables T that have at least two columns and 10-25 rows, resulting in more than 700K tables.…”

Section: Generating Examples From Tablesmentioning

confidence: 99%

See 1 more Smart Citation

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Ori¹,

Talmor²,

Berant³

2021

Preprint

View full text Add to dashboard Cite

Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to leverage semi-structured tables, and automatically generate at scale questionparagraph pairs, where answering the question requires reasoning over multiple facts in the paragraph. We add a pre-training step over this synthetic data, which includes examples that require 16 different reasoning skills such as number comparison, conjunction, and fact composition. To improve data efficiency, we propose sampling strategies that focus training on reasoning skills the model is currently lacking. We evaluate our approach on three reading comprehension datasets that are focused on reasoning, and show that our model, PReasM, substantially outperforms T5, a popular pre-trained encoder-decoder model. Moreover, sampling examples based on current model errors leads to faster training and higher overall performance.

show abstract

Section: Related Workmentioning

confidence: 97%

Section: Generating Examples From Tablesmentioning

confidence: 99%

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Ori¹,

Talmor²,

Berant³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To isolate rows from a premise table that are related to the hypothesis sentence, we apply Distracting Rows Removal (DRR), which was proposed by the previous approach (Neeraja et al, 2021). Since that approach was NN-based, a sentence vector representation was generated for each row in the table, and the original DRR was applied to the sentence representation.…”

Section: Rows Filteringmentioning

confidence: 99%

“…In recent years, modern neural network (NN) approaches have achieved high performance in many Natural Language Understanding benchmarks, such as BERT (Devlin et al, 2019). NNbased approaches (Neeraja et al, 2021) have also achieved high accuracy on the NLI task between semi-structured tables and texts, but previous studies have questioned whether NN-based models truly understand the various linguistic phenomena (Jia and Liang, 2017;Naik et al, 2018;Rozen et al, 2019;Ravichander et al, 2019;Richardson et al, 2020). These studies have shown that NN-based approaches have failed to achieve a high performance in numerical reasoning.…”

Section: Introductionmentioning

confidence: 99%

Logical Inference for Counting on Semi-structured Tables

Kurosawa¹,

Yanaka²

2022

Preprint

View full text Add to dashboard Cite

Recently, the Natural Language Inference (NLI) task has been studied for semistructured tables that do not have a strict format. Although neural approaches have achieved high performance in various types of NLI, including NLI between semi-structured tables and texts, they still have difficulty in performing a numerical type of inference, such as counting. To handle a numerical type of inference, we propose a logical inference system for reasoning between semi-structured tables and texts. We use logical representations as meaning representations for tables and texts and use model checking to handle a numerical type of inference between texts and tables. To evaluate the extent to which our system can perform inference with numerical comparatives, we make an evaluation protocol that focuses on numerical understanding between semi-structured tables and texts in English. We show that our system can more robustly perform inference between tables and texts that requires numerical understanding compared with current neural approaches.

show abstract

“…IN-FOTABS (Gupta et al, 2020) and TABFACT focus on verifying a statement given a table from Wikipedia 1 as evidence. Neeraja et al (2021) propose simple modifications to how information is presented to existing textual models such as RoBERTa (Liu et al, 2019) to improve tabular fact verification. Along with releasing TABFACT, Chen et al (2020) also discuss two promising approaches for tabular fact verification, Latent Program Algorithm (LPA) and Table-BERT.…”

Section: Introductionmentioning

confidence: 99%

Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning

Gautam¹,

Gupta²,

Shrivastava³

2021

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)

View full text Add to dashboard Cite

Tables are widely used in various kinds of documents to present information concisely. Understanding tables is a challenging problem that requires an understanding of language and table structure, along with numerical and logical reasoning. In this paper, we present our systems to solve Task 9 of SemEval-2021: Statement Verification and Evidence Finding with Tables (SEM-TAB-FACTS). The task consists of two subtasks: (A) Given a table and a statement, predicting whether the table supports the statement and (B) Predicting which cells in the table provide evidence for/against the statement. We fine-tune TAPAS (a model which extends BERT's architecture to capture tabular structure) for both the subtasks as it has shown state-of-the-art performance in various table understanding tasks. In subtask A, we evaluate how transfer learning and standardizing tables to have a single header row improves TAPAS' performance. In subtask B, we evaluate how different fine-tuning strategies can improve TAPAS' performance. Our systems achieve an F1 score of 67.34 in subtask A three-way classification, 72.89 in subtask A two-way classification, and 62.95 in subtask B.

show abstract

Incorporating External Knowledge to Enhance Tabular Reasoning

Cited by 21 publications

References 21 publications

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Logical Inference for Counting on Semi-structured Tables

Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning

Contact Info

Product

Resources

About