Blind First-Order Perspective Distortion Correction Using Parallel Convolutional Neural Networks

Gallego, Neil Patrick Del; Ilao, Joel P.; Cordel, Macario O.

doi:10.3390/s20174898

Cited by 6 publications

(3 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assume the perceived distortion is correlated to the loss of local conformality when mapping objects from the 3D world to 2D image space [3], and employ a stereographic projection to preserve the local conformality. While some recent approaches use end-to-end deep neural networks for image rectification [40], [41], our method employs a deep neural network for subject segmentation and solves an optimization problem to generate temporally consistent warps.…”

Section: B Distortion Correctionmentioning

confidence: 99%

Correcting Face Distortion in Wide-Angle Videos

Lai,

Shih,

Liang

et al. 2021

Preprint

View full text Add to dashboard Cite

Video blogs and selfies are popular social media formats, which are often captured by wide-angle cameras to show human subjects and expanded background. Unfortunately, due to perspective projection, faces near corners and edges exhibit apparent distortions that stretch and squish the facial features, resulting in poor video quality. In this work, we present a video warping algorithm to correct these distortions.Our key idea is to apply stereographic projection locally on the facial regions. We formulate a mesh warp problem using spatial-temporal energy minimization and minimize background deformation using a line-preservation term to maintain the straight edges in the background. To address temporal coherency, we constrain the temporal smoothness on the warping meshes and facial trajectories through the latent variables. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths. The user study shows that 83.9% of users prefer our algorithm over other alternatives based on perspective projection. The video results can be found at https://www.wslai.net/publications/video_face_correction/.

show abstract

Section: B Distortion Correctionmentioning

confidence: 99%

Correcting Face Distortion in Wide-Angle Videos

Lai,

Shih,

Liang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Perspective deformation is also a common problem in optical imaging [20]- [23]. But optical images are reflection (surface) images and such perspective deformation is typically caused by the distortion of camera lens.…”

Section: Introductionmentioning

confidence: 99%

Learning Perspective Deformation in X-Ray Transmission Imaging

Huang¹,

Maier²,

Fietkau³

et al. 2022

Preprint

View full text Add to dashboard Cite

In cone-beam X-ray transmission imaging, due to the divergence of X-rays, imaged structures with different depths have different magnification factors on an X-ray detector, which results in perspective deformation. Perspective deformation causes difficulty in direct, accurate geometric assessments of anatomical structures. In this work, in order to reduce perspective deformation in X-ray images acquired from regular cone-beam computed tomography (CBCT) systems, we investigate on learning perspective deformation, i.e., converting perspective (cone-beam) projections into orthogonal (parallel-beam) projections. Directly converting a single perspective projection image into an orthogonal projection image is extremely challenging due to the lack of depth information. Therefore, we propose to utilize one additional perspective projection, a complementary (180 • ) or orthogonal (90 • ) view, to provide a certain degree of depth information. Furthermore, learning perspective deformation in different spatial domains, i.e. in Cartesian, polar, and log-polar coordinates, is investigated. Our proposed method is evaluated on numerical spherical bead phantoms as well as patients' chest and head X-ray data. The experiments on numerical bead phantom data demonstrate that learning perspective deformation in polar coordinates has significant advantages over learning in Cartesian coordinates, as root-mean-square error (RMSE) decreases from 5.31 to 1.40, while learning in log-polar coordinates has no further considerable improvement (RMSE = 1.85). In addition, using a complementary view (RMSE = 1.40) is better than an orthogonal view (RMSE = 3.87). The experiments on patients' chest and head data demonstrate that learning perspective deformation using dual complementary views is also applicable in anatomical X-ray data, allowing accurate cardiothoracic ratio measurements in chest X-ray images and cephalometric analysis in synthetic cephalograms from cone-beam X-ray projections. Our proof-of-concept experiments indicate that learning perspective deformation has the potential to empower conventional CBCT systems with more applications, e.g. chest X-ray imaging, which typically require specialized X-ray devices.

show abstract

“…Thermal noise [11] • • environment, electronics Salt and pepper noise [7] • • electronics Random telegraph noise [4] • • electronics Temporal contrast/ brightness inconsistencies [12] • • electronics, environment, software homomorphic filtering [13], stabilization algorithms [14], temporal filtering [12], neural networks [15] Line, stripe, wave and ring artifacts [16,17] • • electronics, environment, optics wavelet/Fourier filtering [10], spatial filtering [16], neural networks [18] Compression artifacts [19] • • software bilateral filtering [8], fuzzy filtering [20] neural networks [19,[21][22][23] Projective distortions [24] • • optics model-based calculations [25], neural networks [26,27] Out-of-focus effects [28,29] • • optics morphological filtering [30], neural networks [31,32] Fixed pattern noise [33,34] • • electronics, environment, optics reference imaging [33], neural networks [35] Aliasing [36] • • software anti-aliasing algorithms [36], neural networks [37] Rolling shutter effects [38] • • electronics neural networks [39] Artifacts are visually recognizable in a variety of shapes and intensities. Table 1 shows common artifact types occurring in sensor images, their sources, and algorithmic example methods which can be used to...…”

Section: Introductionmentioning

confidence: 99%

A Data-Centric Augmentation Approach for Disturbed Sensor Image Segmentation

2021

View full text Add to dashboard Cite

Deep learning methods have become increasingly popular for optical sensor image analysis. They are adaptable to specific tasks and simultaneously demonstrate a high degree of generalization capability. However, applying deep neural networks to problems with low availability of labeled training data can lead to a model being incapable of generalizing to possible scenarios that may occur in test data, especially with the occurrence of dominant imaging artifacts. We propose a data-centric augmentation approach based on generative adversarial networks that overlays the existing labeled data with synthetic artifacts that are generated from data not present in the training set. This augmentation leads to a more robust generalization capability in semantic segmentation. Our method does not need any additional labeling and does not lead to additional memory or time consumption during inference. Further, we find it to be more effective than comparable approaches that are based on procedurally generated disturbances and the direct use of real disturbances. Building upon the improved segmentation results, we observe that our approach leads to improvements of 22% in the F1-score for an evaluated detection problem, which promises significant robustness towards future disturbances. In the context of sensor-based data analysis, the compensation of image artifacts is a challenge. When the structures of interest are not clearly visible in an image, algorithms that can cope with artifacts are crucial for obtaining the desired information. Thereby, the high variation of artifacts, the combination of different types of artifacts, and their similarity to signals of interest are specific issues that have to be considered in the analysis. Despite the high generalization capability of deep learning-based approaches, their recent success was driven by the availability of large amounts of labeled data. Therefore, the provision of comprehensive labeled image data with different characteristics of image artifacts is of importance. At the same time, applying deep neural networks to problems with low availability of labeled data remains a challenge. This work presents a data-centric augmentation approach based on generative adversarial networks that augments the existing labeled data with synthetic artifacts generated from data not present in the training set. In our experiments, this augmentation leads to a more robust generalization in segmentation. Our method does not need additional labeling and does not lead to additional memory or time consumption during inference. Further, we find it to be more effective than comparable augmentations based on procedurally generated artifacts and the direct use of real artifacts. Building upon the improved segmentation results, we observe that our approach leads to improvements of 22% in the F1-score for an evaluated detection problem. Having achieved these results with an example sensor, we expect increased robustness against artifacts in future applications.

show abstract

Blind First-Order Perspective Distortion Correction Using Parallel Convolutional Neural Networks

Cited by 6 publications

References 48 publications

Correcting Face Distortion in Wide-Angle Videos

Correcting Face Distortion in Wide-Angle Videos

Learning Perspective Deformation in X-Ray Transmission Imaging

A Data-Centric Augmentation Approach for Disturbed Sensor Image Segmentation

Contact Info

Product

Resources

About