TableNet: Deep Learning Model for End-to-end Table Detection and Tabular Data Extraction from Scanned Document Images

Abstract

With the widespread use of mobile phones and scanners to photograph and upload documents, the need for extracting the information trapped in unstructured document images such as retail receipts, insurance claim forms and financial invoices is becoming more acute. A major hurdle to this objective is that these images often contain information in the form of tables and extracting data from tabular sub-images presents a unique set of challenges. This includes accurate detection of the tabular region within an image, and subsequently detecting and extracting information from the rows and columns of the detected table. While some progress has been made in table detection, extracting the table contents is still a challenge since this involves more fine grained table structure(rows & columns) recognition. Prior approaches have attempted to solve the table detection and structure recognition problems independently using two separate models. In this paper, we propose TableNet: a novel end-toend deep learning model for both table detection and structure recognition. The model exploits the interdependence between the twin tasks of table detection and table structure recognition to segment out the table and column regions. This is followed by semantic rule-based row extraction from the identified tabular sub-regions. The proposed model and extraction approach was evaluated on the publicly available ICDAR 2013 and Marmot Table datasets obtaining state of the art results. Additionally, we demonstrate that feeding additional semantic features further improves model performance and that the model exhibits transfer learning across datasets. Another contribution of this paper is to provide additional table structure annotations for the Marmot data, which currently only has annotations for table detection.

HybridTabNet: Towards Better Table Detection in Scanned Document Images

“…Thus far, we have seen a similar trend in the advancement of object detection algorithms in computer vision [14][15][16][17] with the progress in table detection systems [6][7][8]11,12]. Although recent object detection frameworks have noticeably improved the performance of table detection approaches [7,18], there is room in further reducing the close false positives. These case of close false positives can be resolved by leveraging the instance segmentation networks where an additional segmentation loss is added along with the bounding box and classification loss [12,17,19].…”

HybridTabNet: Towards Better Table Detection in Scanned Document Images

“…So far, we have seen a similar trend in the advancement of object detection algorithms in computer vision [14][15][16][17] with the progress in table detection systems [6][7][8]11,12]. Although recent object detection frameworks have noticeably improved the performance of table detection approaches [7,18], there is a room in further reducing the close false positives. These case of close false positives can be resolved by leveraging the instance segmentation networks where an additional segmentation loss is added along with the bounding box and classification loss [12,17,19].…”

Rescuing historical climate observations to support hydrological research

“…Following the launch of the ICDAR (International Conference on Document Analysis and Recognition) table competition [3] in 2013, there have been several deep learning approaches to help solve the problem of table detection and table structure recognition. TableNet [4], a deep learning solution that detects the table regions in an image and subsequently detects the columns in the table, uses a rule-based approach to extract the text contained in the cells of the detected table. CascadeTabNet [5] is an end-to-end deep learning approach to table detection and table structure recognition using the Cascade Mask R-CNN HRNet model.…”