PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer

Abstract

Most research on pseudo relevance feedback (PRF) has been done in vector space and probabilistic retrieval models. This paper shows that Transformer-based rerankers can also benefit from the extra context that PRF provides. It presents PGT, a graph-based Transformer that sparsifies attention between graph nodes to enable PRF while avoiding the high computational complexity of most Transformer architectures. Experiments show that PGT improves upon non-PRF Transformer reranker, and it is at least as accurate as Transformer PRF models that use full attention, but with lower computational costs.

A semantically enhanced text retrieval framework with abstractive summarization

“…Pan et al 18 integrated kernel co-occurrence information into Rocchio and RM3 and proposed KRoc and KRM3 to achieve improved retrieval performance. Recently, Yu et al 19 proposed PGT, a feedback method based on transformer architecture. In PGT, pseudo-relevance feedback is performed by feeding each feedback document and the target document into BERT in series.…”