Towards an Efficient Semantic Segmentation Method of ID Cards for Verification Systems

Abstract

Removing the background in ID Card images is a real challenge for remote verification systems because many of the re-digitalised images present cluttered backgrounds, poor illumination conditions, distortion and occlusions. The background in ID Card images confuses the classifiers and the text extraction. Due to the lack of available images for research, this field represents an open problem in computer vision today. This work proposes a method for removing the background using semantic segmentation of ID Cards. In the end, images captured in the wild from real operation, using a manually labelled dataset consisting of 45,007 images, with five types of ID Cards from three countries (Chile, Argentina and Mexico), including typical presentation attack scenarios, were used. This method can help to improve the following stages in a regular identity verification or document tampering detection system. Two Deep Learning approaches were explored, based on MobileUNet and DenseNet10. The best results were obtained using MobileUNet, with 6.5 million parameters. A Chilean ID Card's mean Intersection Over Union (IoU) was 0.9926 on a private test dataset of 4,988 images. The best results for the fused multi-country dataset of ID Card images from Chile, Argentina and Mexico reached an IoU of 0.9911. The proposed methods are lightweight enough to be used in real-time operation on mobile devices.

Synthetic ID Card Image Generation for Improving Presentation Attack Detection

“…This is replicated using a random protective transformation. Finally, we confirm that the background of the image is entirely black (R=0, G=0, B=0), resembling the result from the segmentation network[17].Since the face image dataset contains 2,069 images, we decided to create synthetic ID cards in batches of this number. In this way, every batch will have the same faces in the same order but with different signatures, text fields, colours and perspectives.…”

“…al. [17] in order to remove the background pixels and force the frauddetection network to learn features only from the pixels inside of the ID card. Figure 1 show examples of the segmented ID card images.…”

Improving Presentation Attack Detection for ID Cards on Remote Verification Systems

“…Finally, a colour space conversion back to RGB is performed. A detailed explanation of the segmentation algorithm can be found in [20].…”