ScanBank: A Benchmark Dataset for Figure Extraction from Scanned Electronic Theses and Dissertations

Abstract

We focus on electronic theses and dissertations (ETDs), aiming to improve access and expand their utility, since more than 6 million are publicly available, and they constitute an important corpus to aid research and education across disciplines. The corpus is growing as new born-digital documents are included, and since millions of older theses and dissertations have been converted to digital form to be disseminated electronically in institutional repositories. In ETDs, as with other scholarly works, figures and tables can communicate a large amount of information in a concise way. Although methods have been proposed for extracting figures and tables from born-digital PDFs, they do not work well with scanned ETDs. Considering this problem, our assessment of state-of-the-art figure extraction systems is that the reason they do not function well on scanned PDFs is that they have only been trained on born-digital documents. To address this limitation, we present ScanBank, a new dataset containing 10 thousand scanned page images, manually labeled by humans as to the presence of the 3.3 thousand figures or tables found therein. We use this dataset to train a deep neural network model based on YOLOv5 to accurately extract figures and tables from scanned ETDs. We pose and answer important research questions aimed at finding better methods for figure extraction from scanned documents. One of those concerns the value for training, of data augmentation techniques applied to born-digital documents which are used to train models better suited for figure extraction from scanned documents. To the best of our knowledge, ScanBank is the first manually annotated dataset for figure and table extraction for scanned ETDs. A YOLOv5-based model, trained on ScanBank, outperforms existing comparable open-source and freely available baseline methods by a considerable margin.

Figure and Figure Caption Extraction for Mixed Raster and Vector PDFs: Digitization of Astronomical Literature with OCR Features

“…Typical modern methods rely on deep learning techniques to detect layout elements on pages [5], often in combination with heuristics [38]. Methods span the range of object detection using models like YOLO [32,38,47] to, more recently, Faster R-CNN [16,34,37,43,49] and Mask-CNN [2,18,26]. Additionally, several pixel-by-pixel segmentation models have been proposed using semantic segmentation [46] and fully convolutional networks [22,24], including "fully convolutional instance segmentation" [13,14,27].…”

“…Table 3 shows how other deep learning models fair on our final test dataset. Here, we use ScanBank [21,47] (based on DeepFigures [38] and trained on a corpus of pre-digital electronic thesis and dissertations (ETDs)) and a version of detectron2 [45] trained on the PubLayNet dataset [52]. ScanBank and detectron2 are used for comparison as they are applied to raster-formatted articles (as opposed to vector-based methods like pdffigures2 [11] which, applied to our data, results in F1 scores of <15% in feature and final test splits).…”

“…Deep learning methods have become popular recently for vector and raster documents [5,38], including those that use methods of semantic segmentation [46] and object detection [35]. While these methods are vital to the extraction of data products from recent academic literature, pre-digital literature is often included in digital platforms with older articles scanned at varying resolutions and deep learning methods developed with newer article training sets often perform poorly on this pre-digital literature [47]. Additionally, layouts, fonts, and article styles are typically different for historical documents when compared to "born-digital" scientific literature [47].…”

“…While these methods are vital to the extraction of data products from recent academic literature, pre-digital literature is often included in digital platforms with older articles scanned at varying resolutions and deep learning methods developed with newer article training sets often perform poorly on this pre-digital literature [47]. Additionally, layouts, fonts, and article styles are typically different for historical documents when compared to "born-digital" scientific literature [47]. In these cases, text extraction must be performed with optical character recognition (OCR), and figures and tables are extracted from the raw OCR results.…”

“…In these cases, text extraction must be performed with optical character recognition (OCR), and figures and tables are extracted from the raw OCR results. When applied to raster-PDF's with text generated from OCR, deep learning document layout analysis methods trained with newer or vector-based PDFs are often not as robust [46,47]. While progress has been made in augmenting these methods for OCR'd pages, especially for electronic theses and dissertations (ETDs) [47], much work can still be done to extract layout elements from these older, raster-based documents.…”