Multi-Layout Invoice Document Dataset (MIDD): A Dataset for Named Entity Recognition

Abstract

The day-to-day working of an organization produces a massive volume of unstructured data in the form of invoices, legal contracts, mortgage processing forms, and many more. Organizations can utilize the insights concealed in such unstructured documents for their operational benefit. However, analyzing and extracting insights from such numerous and complex unstructured documents is a tedious task. Hence, the research in this area is encouraging the development of novel frameworks and tools that can automate the key information extraction from unstructured documents. However, the availability of standard, best-quality, and annotated unstructured document datasets is a serious challenge for accomplishing the goal of extracting key information from unstructured documents. This work expedites the researcher’s task by providing a high-quality, highly diverse, multi-layout, and annotated invoice documents dataset for extracting key information from unstructured documents. Researchers can use the proposed dataset for layout-independent unstructured invoice document processing and to develop an artificial intelligence (AI)-based tool to identify and extract named entities in the invoice documents. Our dataset includes 630 invoice document PDFs with four different layouts collected from diverse suppliers. As far as we know, our invoice dataset is the only openly available dataset comprising high-quality, highly diverse, multi-layout, and annotated invoice documents.

An Overview of Data Extraction From Invoices

“…This system uses connected components and pixel analysis for classifying elements such as paragraphs, graphics, images, and tables in the document. In [11] the authors propose a dataset for unstructured invoice documents that covers a wide range of layouts, which is designed to generalize key field extraction tasks for unstructured documents. The dataset is evaluated using various feature extraction techniques as well as Artificial Intelligence methods.…”

“…Let us particularly mention the works of Alfonseca et al [3] and R. Evans [40] that use the notion of "open domain". Recently, data sets have been made available for NER related to invoices [11]. From a practical point of view, Mikolov et al [90] demonstrate the benefit of using vector representation of words and also that it is possible to train a model of neural networks on a large training set, including a large number of sentences with approximately one billion words and a vocabulary of more than one million different words.…”

“…Topic [18] open source OCR solution [89] handwritten character recognition [28] handwritten OCR [96] text recognition using deep learning [62] deep learning based OCR [92] OCR solution including image to speech transformation [98] benchmark sets for OCR [44] OpenCV system [158] neural network based OCR [58] description of Icdar2019 competition on scanned receipt [10] A survey into OCR specialized for medical reports. [77] A technique based on transformer architecture for OCR and a benchmark with modern solutions Reference Topic [133] seminal work on data extraction [7] computational-geometry algorithms for analyzing document structures [2] handling multiple types of data structures [146] considering relations between data [131] orientation of documents [16] document layout analysis [11] data sets for evaluation [81] seminal work on pdf documents management [48] data extraction from tables [113] table extraction for pdf documents [41] table detection for multipage pdf documents [24] solving of the maximum independent set of rectangles problem [149] pdf2table : method for extracting table [46] graph neural network for extracting tables from pdf documents [152] deep learning for pdf table extraction [104] presentation of TAO for table detection and extraction Reference Topic [85] seminal work on NER [135] NER Challenge at CoNLL [35] ACE program : challenge for NER systems [20] empirical study of NER [3] procedure to automatically extend an ontology with domain specific knowledge [40] system for NER in the open domain…”

Automated invoice data extraction using image processing

“…The existence of a dataset in order to train the network for segmentation is another problem in the case of training networks for extracting relevant information from structured documents. Baviskar et al [14] Provides a well annotated dataset which helps in training a network which can recognize varying invoices of the same format. However, in order to train a model which can recognize a wide range of invoices, the network should be trained across a large number of invoices as well.…”