Document image content inventories

Abstract

We report an investigation into strategies, algorithms, and software tools for document image content extraction and inventory, that is, the location and measurement of regions containing handwriting, machine-printed text, photographs, blank space, etc. We have developed automatically trainable methods, adaptable to many kinds of documents represented as bilevel, greylevel, or color images, that offer a wide range of useful tradeoffs of speed versus accuracy using methods for exact and approximate k-Nearest Neighbor classification. We have adopted a policy of classifying each pixel (rather than regions) by content type: we discuss the motivation and engineering implications of this choice. We describe experiments on a wide variety of document-image and content types, and discuss performance in detail in terms of classification speed, per-pixel classification accuracy, per-page inventory accuracy, and subjective quality of page segmentation. These show that even modest perpixel classification accuracies (of, e.g., 60-70%) support usefully high recall and precision rates (of, e.g., 80-90%) for retrieval queries of document collections seeking pages that contain a given minimum fraction of a certain type of content.

Document Content Extraction Using Automatically Discovered Features

“…Detail description of these features can be found in [3]. The error rate for this set is 13.6% and is shown in Fig.…”

Truthing for Pixel-Accurate Segmentation

“…Our classifier, discussed in [4,6,7], is an approximation of kNearest Neighbors and is used to classify each pixel in a document image by assigning it a class label, such as machine print, handwriting, photograph, etc. Features are extracted from each training sample (pixel) from a small, local window of no more than 20 pixels wide.…”

“…In previous work [4,5,9,10, 1], we have described a research program investigating versatile algorithms for document image content extraction, that is locating regions containing machine printed text, handwriting, photographs, etc. This program seeks to solve this problem in full generality, handling a vast variety of document and image types.…”

Text versus non-text distinction in online handwritten documents

“…The bottom-up strategy, on the other hand, performs classification on small, naturally given parts of a document e.g. pixels, connected components, or individual strokes in online documents [2,8,18]. A clustering algorithm may follow to group small entities into larger, meaningful segments.…”

“…Top-down methods are prevailing if the document structure can be analyzed rather easily [9] (as in scientific papers or newspapers, for example). Pixel classification is preferred where the structure is difficult to recognize [2] (as in magazines where text and images may be mixed rather irregularly). In the field of online handwritten document analysis, the distinction of text and non-text is accomplished with a bottom-up approach in [8,13] where single strokes, as the smallest entities, are classified.…”