ID documents matching and localization with multi-hypothesis constraints

Chiron, Guillaume; Ghanmi, Nabil; Awal, Ahmad Montaser

doi:10.1109/icpr48806.2021.9412437

Cited by 5 publications

(6 citation statements)

References 25 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The typical base features are keypoints and their descriptors, however the feature space can be expanded with line segments, quadrangles, vanishing points, etc. [11,43]. [15]; f) result of [16]; (g) source image; (h) result of [15]; (i) result of [16];…”

Section: Feature-based Document Location and Identificationmentioning

confidence: 99%

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

et al. 2022

View full text Add to dashboard Cite

Identity documents recognition is an important sub-field of document analysis, which deals with tasks of robust document detection, type identification, text fields recognition, as well as identity fraud prevention and document authenticity validation given photos, scans, or video frames of an identity document capture. Significant amount of research has been published on this topic in recent years, however a chief difficulty for such research is scarcity of datasets, due to the subject matter being protected by security requirements. A few datasets of identity documents which are available lack diversity of document types, capturing conditions, or variability of document field values. In this paper, we present a dataset MIDV-2020 which consists of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock identity documents, each with unique text field values and unique artificially generated faces, with rich annotation. The dataset contains 72409 annotated images in total, making it the largest publicly available identity document dataset to the date of publication. We describe the structure of the dataset, its content and annotations, and present baseline experimental results to serve as a basis for future research. For the task of document location and identification content-independent, feature-based, and semantic segmentation-based methods were evaluated. For the task of document text field recognition, the Tesseract system was evaluated on field and character levels with grouping by field alphabets and document types. For the task of face detection, the performance of Multi Task Cascaded Convolutional Neural Networks-based method was evaluated separately for different types of image input modes. The baseline evaluations show that the existing methods of identity document analysis have a lot of room for improvement given modern challenges. We believe that the proposed dataset will prove invaluable for advancement of the field of document analysis and recognition.

show abstract

Section: Feature-based Document Location and Identificationmentioning

confidence: 99%

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…On the MIDV500 dataset, an IoU of 0.9830 with all 4 vertices within the frame was obtained. In [39], a method for spotting and locating identity documents in the wild is presented, using a priori information along with a list of predefined models, using specific ID document features. For solving the problem, the approach tests different crop hypotheses, competing between them, to select at least one candidate that correctly crops the document, using a custom visual similarity metric.…”

Section: Related Workmentioning

confidence: 99%

Towards an Efficient Semantic Segmentation Method of ID Cards for Verification Systems

Lara¹,

Valenzuela²,

Schulz³

et al. 2021

Preprint

View full text Add to dashboard Cite

Removing the background in ID Card images is a real challenge for remote verification systems because many of the re-digitalised images present cluttered backgrounds, poor illumination conditions, distortion and occlusions. The background in ID Card images confuses the classifiers and the text extraction. Due to the lack of available images for research, this field represents an open problem in computer vision today. This work proposes a method for removing the background using semantic segmentation of ID Cards. In the end, images captured in the wild from real operation, using a manually labelled dataset consisting of 45,007 images, with five types of ID Cards from three countries (Chile, Argentina and Mexico), including typical presentation attack scenarios, were used. This method can help to improve the following stages in a regular identity verification or document tampering detection system. Two Deep Learning approaches were explored, based on MobileUNet and DenseNet10. The best results were obtained using MobileUNet, with 6.5 million parameters. A Chilean ID Card's mean Intersection Over Union (IoU) was 0.9926 on a private test dataset of 4,988 images. The best results for the fused multi-country dataset of ID Card images from Chile, Argentina and Mexico reached an IoU of 0.9911. The proposed methods are lightweight enough to be used in real-time operation on mobile devices.

show abstract

“…2 clearly demonstrate that the proposed method outperforms other systems which do not take into account the document content. The system proposed in [27] outperforms our system and generate the baseline for a document outer border detection with the knowledge of the inner structure of the document.…”

Section: Discussionmentioning

confidence: 93%

“…The first four columns represent the accuracy measurements on MIDV-500 for 4 document localization systems. All of these systems, except for [27], do not take into account the document content. A brief description of the MIDV-500 subsets is given in the third column of the table; a detailed description, as well as the subsets themselves, can be found in the supplementary materials.…”

Section: Results Of Experiments On Midv-500mentioning

confidence: 99%

See 1 more Smart Citation

Advanced Hough-based method for on-device document localization

et al. 2021

View full text Add to dashboard Cite

The demand for on-device document recognition systems increases in conjunction with the emergence of more strict privacy and security requirements. In such systems, there is no data transfer from the end device to a third-party information processing servers. The response time is vital to the user experience of on-device document recognition. Combined with the unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on consumer-grade end devices such as smartphones, the time limitations put significant constraints on the computational complexity of the applied algorithms for on-device execution. In this work, we consider document location in an image without prior knowledge of the docu-ment content or its internal structure. In accordance with the published works, at least 5 systems offer solutions for on-device document location. All these systems use a location method which can be considered Hough-based. The precision of such systems seems to be lower than that of the state-of-the-art solutions which were not designed to account for the limited computational resources. We propose an advanced Hough-based method. In contrast with other approaches, it accounts for the geometric invariants of the central projection model and combines both edge and color features for document boundary detection. The proposed method allowed for the second best result for SmartDoc dataset in terms of precision, surpassed by U-net like neural network. When evaluated on a more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best precision compared to published methods. Our method retained the applicability to on-device computations.

show abstract

ID documents matching and localization with multi-hypothesis constraints

Cited by 5 publications

References 25 publications

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

Towards an Efficient Semantic Segmentation Method of ID Cards for Verification Systems

Advanced Hough-based method for on-device document localization

Contact Info

Product

Resources

About