Complex Document Classification and Localization Application on Identity Document Images

Abstract

This paper studies the problem of document image classification. More specifically, we address the classification of documents composed of few textual information and complex background (such as identity documents). Unlike most existing systems, the proposed approach simultaneously locates the document and recognizes its class. The latter is defined by the document nature (passport, ID, etc.), emission country, version, and the visible side (main or back). This task is very challenging due to unconstrained capturing conditions, sparse textual information, and varying components that are irrelevant to the classification, e.g. photo, names, address, etc.First, a base of document models is created from reference images. We show that training images are not necessary and only one reference image is enough to create a document model. Then, the query image is matched against all models in the base. Unknown documents are rejected using an estimated quality based on the extracted document. The matching process is optimized to guarantee an execution time independent from the number of document models. Once the document model is found, a more accurate matching is performed to locate the document and facilitate information extraction. Our system is evaluated on several datasets with up to 3042 real documents (representing 64 classes) achieving an accuracy of 96.6%.

An Improved Document Image Classification using Deep Transfer Learning and Feature Reduction

“…We also compared our results with available results in other studies in related fields; in [30] an accuracy score of 98.4 %, by combining the use of KNN and AlexNet on invoice classification (on a dataset of 1380 invoices), and in [31]the authors achieved an accuracy score of 96.6% on Identity document classification based on deep learning and document modeling.…”

ID documents matching and localization with multi-hypothesis constraints

“…All-in-one classification and localization solutions are proposed in [4], [19], and [20]. In this context, supported document models are generated using a unique reference image captured in good conditions without any distortion.…”

“…SURF [5]). Additional annotations are added manually such as masks of variable areas and expected crop quadrilateral (which do not always fit physical/visible borders) [4]. The classification is achieved by maximizing the matching score between model keypoints and the query image ones.…”

“…Our solution is built upon a classification approach proposed earlier in [4]. In this context, we show the importance of rejection capacities in a document classification system.…”

An Automatic Reader of Identity Documents

“…In [2] a computer vision technique, based on visual saliency (instead of edge or contour detection) is used to locate identity documents inside a photo or video frame, without any prior knowledge about the document. Another approach is used in [3], where the document is simultaneously located and classified, by extracting a set of keypoints from the image, and comparing them with a previously built database. A similar technique is used in [4], but for classification only purposes.…”