A Gated and Bifurcated Stacked U-Net Module for Document Image Dewarping

Abstract

Capturing images of documents is one of the easiest and most used methods of recording them. These images however, being captured with the help of handheld devices, often lead to undesirable distortions that are hard to remove. We propose a supervised Gated and Bifurcated Stacked U-Net module to predict a dewarping grid and create a distortion free image from the input. While the network is trained on synthetically warped document images, results are calculated on the basis of real world images. The novelty in our methods exists not only in a bifurcation of the U-Net to help eliminate the intermingling of the grid coordinates, but also in the use of a gated network which adds boundary and other minute line level details to the model. The end-to-end pipeline proposed by us achieves state-of-the-art performance on the DocUNet dataset after being trained on just 8 percent of the data used in previous methods.

RectiNet-v2: A stacked network architecture for document image dewarping

“…From Table II and Table I, we infer that our method outperforms methods proposed by [1], [2] and [4] in MS-SSIM while outperforming the method proposed by [3] in SSIM measured at original resolution.…”

“…The end-to-end network proposed in DocUNet consisted of a stacked U-Net architecture as the backbone. The method of data generation proposed by [2] was used by [4] and [1] in their networks.…”

“…We further post-process the dewarped document images as part of our dewarping module with the help of bilateral filtering which smoothens folds and crumples. As a variation to post-processing by [1] which leaves textual regions blurred in some images, we maintain the level of filtering used by monitoring the blur before and after the filter by computing the variance of a Laplacian filter on the image. V. EXPERIMENTS For comparison of the dewarp quality of our methods, metrics like MS-SSIM (Multi Scale Structural Similarity Index), SSIM (Structural Similarity Index), and LD(Local Distortion) are used.…”

“…pixel areas (880x680), while we use original images for the calculation of SSIM along the levels to gauge the quality of dewarp at various levels in Table II. The MS-SSIM values for our methods at the original resolution and comparison with RectiNet by [1] can be found in the supplementary. We tabulate the results of our experiments on the benchmark both before and after the post-processing step in Table I.…”

“…These however, fail when curves and folds occur simultaneously in document images, which require a more indepth and varied analysis. To rectify these complex document images, deep learning methods have been introduced recently by [1], [2], [3] and [4]. These deep learning methods treat the problem of document dewarping as the prediction of a dense grid that can aid in the dewarping process.…”