Passage Retrieval on Structured Documents Using Graph Attention Networks

Abstract

Passage Retrieval systems aim at retrieving and ranking small text units according to their estimated relevance to a query. A usual practice is to consider the context a passage appears in (its containing document, neighbour passages, etc.) to improve its relevance estimation. In this work, we study the use of Graph Attention Networks (GATs), a graph node embedding method, to perform passage contextualization. More precisely, we first propose a document graph representation based on several inter-and intra-document relations. Then, we investigate two ways of leveraging the use of GATs on this representation in order to incorporate contextual information for passage retrieval. We evaluate our approach on a Passage Retrieval task for structured documents: CLEF-IP2013. Our results show that our document graph representation coupled with the expressive power of GATs allows for a better context representation leading to improved performances.

Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks

“…These models have shown their effectiveness and flexibility in a wide variety of NLP tasks, including text classification , relation extraction (Zhang et al, 2018;Carbonell et al, 2020), and question answering (Cao et al, 2019;Xu et al, 2021b). Recently, GNNs have been employed for document retrieval to enhance the vector representations by leveraging the topological structure of the documents, where nodes are passages from a document and edges are relations between these passages (Xu et al, 2021a;Zhang et al, 2021b;Albarede et al, 2022).…”

Université Grenoble Alpes / LIG at TREC Deep Learning Track 2022

“…Document retrieval (BM25) is applied as a first stage retrieval returning the top 1000 documents. Every passage from these documents is then reranked using HGATs to leverage its relations with other passages in a graph-based document representation [1]. Passage and query embeddings are computed using a multiple-representation embedding encoder similar to the one in the Colbert architecture [2].…”