FeTaQA: Free-form Table Question Answering

Nan, Linyong; Hsieh, Chia‐Chun; Mao, Ziming; Lin, Xi Victoria; Verma, Neha; Zhang, Rui; Kryściński, Wojciech; Schoelkopf, Nick; Kong, Riley; Tang, Xiangru; Mutuma, Mutethia; Rosand, Ben; Trindade, Isabel; Bandaru, Renusree; Cunningham, Jacob; Xiong, Caiming; Radev, Dragomir

doi:10.1162/tacl_a_00446

Cited by 24 publications

(17 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• FetaQA [27] This dataset contains 10,330 clean tables from Wikipedia and 2003 test questions. The questions are generally longer and more complex than those from NQ-Tables.…”

Section: Discussionmentioning

confidence: 99%

Data Discovery using Natural Language Questions via a Self-Supervised Approach

Wang¹,

Fernandez²

2023

Preprint

View full text Add to dashboard Cite

Data discovery systems help users identify relevant data among large table collections. Users express their discovery needs with a program or a set of keywords. Users may express complex queries using programs but it requires expertise. Keyword search is accessible to a larger audience but limits the types of queries supported. An interesting approach is learned discovery systems which find tables given natural language questions. Unfortunately, these systems require a training dataset for each table collection. And because collecting training data is expensive, this limits their adoption.In this paper, we introduce a self-supervised approach to assemble training datasets and train learned discovery systems without human intervention. It requires addressing several challenges, including the design of self-supervised strategies for data discovery, table representation strategies to feed to the models, and relevance models that work well with the synthetically generated questions. We combine all the above contributions into a system, S2LD, that solves the problem end to end. The evaluation results demonstrate the new techniques outperform state-of-the-art approaches on wellknown benchmarks. All in all, the technique is a stepping stone towards building learned discovery systems. The code is open-sourced at https://github.com/TheDataStation/open_table_discovery.

show abstract

“…• FetaQA [27] This dataset contains 10,330 clean tables from Wikipedia and 2003 test questions. The questions are generally longer and more complex than those from NQ-Tables.…”

Section: Discussionmentioning

confidence: 99%

Data Discovery using Natural Language Questions via a Self-Supervised Approach

Wang¹,

Fernandez²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…SQuAD (Rajpurkar et al, 2016) and CNN/Daily Mail (Hermann et al, 2015) are classic datasets for textual data. Table/KB QA datasets mainly focus on structured tables (Pasupat and Liang, 2015;Zhong et al, 2017;Yu et al, 2018;Nan et al, 2022) and knowledge bases (Berant et al, 2013;Yih et al, 2015;Talmor and Berant, 2018;Xie et al, 2022). And some recent works focus on reasoning over more complex tables including hierarchical tables (Cheng et al, 2021b 2021).…”

Section: Related Workmentioning

confidence: 99%

MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data

Zhao¹,

Li²,

Li³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Self Cite

View full text Add to dashboard Cite

Numerical reasoning over hybrid data containing both textual and tabular content (e.g., financial reports) has recently attracted much attention in the NLP community. However, existing question answering (QA) benchmarks over hybrid data only include a single flat table in each document and thus lack examples of multistep numerical reasoning across multiple hierarchical tables. To facilitate data analytical progress, we construct a new large-scale benchmark, MULTIHIERTT, with QA pairs over Multi Hierarchical Tabular and Textual data. MULTIHIERTT is built from a wealth of financial reports and has the following unique characteristics: 1) each document contain multiple tables and longer unstructured texts; 2) most of tables contained are hierarchical; 3) the reasoning process required for each question is more complex and challenging than existing benchmarks; and 4) fine-grained annotations of reasoning processes and supporting facts are provided to reveal complex numerical reasoning. We further introduce a novel QA model termed MT2Net, which first applies facts retrieving to extract relevant supporting facts from both tables and text and then uses a reasoning module to perform symbolic reasoning over retrieved facts. We conduct comprehensive experiments on various baselines. The experimental results show that MULTIHIERTT presents a strong challenge for existing baselines whose results lag far behind the performance of human experts. The dataset and code are publicly available at https://github. com/psunlpgroup/MultiHiertt.

show abstract

“…While it is domain-specific, the included tables have a very peculiar structure (with table rows containing entire natural language sentences that have been split into columns), which in our experience is not representative of tables appearing in most domains. Recently, Nan et al (2022) proposed FeTaQA; another Wikipedia-based dataset but with answers that are long free-form sentences (instead of short answers found in prior datasets).…”

Section: Related Workmentioning

confidence: 99%

“…Experimental evaluation of state-of-the-art (Pasupat and Liang, 2015) TabMCQ (Jauhar et al, 2016) 2016 (Science Exam) WikiSQL (Zhong et al, 2017) 2017 FeTaQA (Nan et al, 2022) 2021…”

Section: Introductionmentioning

confidence: 99%