A Span Extraction Approach for Information Extraction on Visually-Rich Documents

Abstract

Information extraction (IE) for visually-rich documents (VRDs) has achieved SOTA performance recently thanks to the adaptation of Transformer-based language models, which shows the great potential of pre-training methods. In this paper, we present a new approach to improve the capability of language model pre-training on VRDs. Firstly, we introduce a new query-based IE model that employs span extraction instead of using the common sequence labeling approach. Secondly, to extend the span extraction formulation, we propose a new training task focusing on modelling the relationships among semantic entities within a document. This task enables target spans to be extracted recursively and can be used to pre-train the model or as an IE downstream task. Evaluation on three datasets of popular business documents (invoices, receipts) shows that our proposed method achieves significant improvements compared to existing models. The method also provides a mechanism for knowledge accumulation from multiple downstream IE tasks.

Value Retrieval with Arbitrary Queries for Form-like Documents

“…Majumder et al 2020 present a field-value pairing framework that learns the representations of fields and value candidates in the same feature space using metric learning. Nguyen et al 2021 propose a span extraction approach to extract the start and end of a value for each queried field. Gao et al 2021 introduce a field extraction system that can be trained with large-scale unlabeled documents.…”

“…Our baseline. We implement our baseline following (Majumder et al, 2020;Nguyen et al, 2021). Unlike our method that utilizes a unified transformer to deeply model interactions among the query words and the OCR words, our baseline models the interactions in a shallower way (see Section A for details).…”