Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning

Abstract

Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains. However, this idea is less explored for spatial language. We provide two new data resources on multiple spatial language processing tasks. The first dataset is synthesized for transfer learning on spatial question answering (SQA) and spatial role labeling (SpRL). Compared to previous SQA datasets, we include a larger variety of spatial relation types and spatial expressions. Our data generation process is easily extendable with new spatial expression lexicons. The second one is a real-world SQA dataset with humangenerated questions built on an existing corpus with SPRL annotations. This dataset can be used to evaluate spatial language processing models in realistic situations. We show pretraining with automatically generated data significantly improves the SOTA results on several SQA and SPRL benchmarks, particularly when the training data in the target domain is small.

Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark

“…Both of these datasets emphasize directional spatial relations (Cohn and Hazarika 2001;Skiadopoulos and Koubarakis 2001;Cohn and Renz 2008;Chen et al 2015). Three spatial QA datasets: SpartQA(Mirzaee and Rajaby 2021), SPARTUN, and RESQ (Mirzaee and Kordjamshidi 2022) expanded the resource landscape by encompassing wider-ranging spatial language expressions, posing challenges for traditional logical programming, and are important benchmarks for evaluating LLMs' spatial reasoning capabilities.…”