Multiple Selection Extrapolation for improved spatial error concealment

2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP)

Brand

Schober

2021

Self Cite

Quarter sampling and three-quarter sampling are novel sensor concepts that enable the acquisition of higher resolution images without increasing the number of pixels. This is achieved by non-regularly covering parts of each pixel of a lowresolution sensor such that only one quadrant or three quadrants of the sensor area of each pixel is sensitive to light. Combining a properly designed mask and a high-quality reconstruction algorithm, a higher image quality can be achieved than using a low-resolution sensor and subsequent upsampling. For the latter case, the image quality can be further enhanced using super resolution algorithms such as the very deep super resolution network (VDSR). In this paper, we propose a novel end-to-end neural network to reconstruct high resolution images from nonregularly sampled sensor data. The network is a concatenation of a locally fully connected reconstruction network (LFCR) and a standard VDSR network. Altogether, using a three-quarter sampling sensor with our novel neural network layout, the image quality in terms of PSNR for the Urban100 dataset can be increased by 2.96 dB compared to the state-of-the-art approach. Compared to a low-resolution sensor with VDSR, a gain of 1.11 dB is achieved.

Section: Introductionmentioning

confidence: 99%

A Novel End-To-End Network for Reconstruction of Non-Regularly Sampled Image Data Using Locally Fully Connected Layers

2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP)

Brand

Schober

2021

Self Cite

“…For a proper reconstruction, high-quality reconstruction algorithms such as the frequency selective reconstruction (FSR) [5] need to be used in combination with optimized quarter sampling patterns such as those in [6]. FSR has shown to be a successful reconstruction scheme for various inpainting and extrapolation tasks [7,8,9] and showed best results for nonregular sampling and quarter sampling in [1,5,6].…”

Section: Introductionmentioning

confidence: 99%

Enhanced Image Reconstruction From Quarter Sampling Measurements Using An Adapted Very Deep Super Resolution Network

2020 IEEE International Conference on Image Processing (ICIP)

Fischer

Brand

et al. 2020

Self Cite

Quarter sampling is a novel sensor concept that enables the acquisition of higher resolution images without increasing the number of pixels. This is achieved by covering three quarters of each pixel of a low-resolution sensor such that only one quadrant of the sensor area of each pixel is sensitive to light. By randomly masking different parts, effectively a non-regular sampling of a higher resolution image is performed. Combining a properly designed mask and a high-quality reconstruction algorithm, a higher image quality can be achieved than using a low-resolution sensor and subsequent upsampling. For the latter case, the image quality can be enhanced using super resolution algorithms. Recently, algorithms based on machine learning such as the Very Deep Super Resolution network (VDSR) proofed to be successful for this task. In this work, we transfer the concepts of VDSR to the special case of quarter sampling. Besides adapting the network layout to take advantage of the case of quarter sampling, we introduce a novel data augmentation technique enabled by quarter sampling. Altogether, using the quarter sampling sensor, the image quality in terms of PSNR can be increased by +0.67 dB for the Urban 100 dataset compared to using a low-resolution sensor with VDSR.

“…In terms of the reconstruction algorithm we investigate linear interpolation and nearest neighbor interpolation, as well as two more sophisticated algorithms, namely, steering kernel regression (SKR) [5] and frequency selective reconstruction (FSR) [6]. FSR has shown to be a successful reconstruction scheme for various inpainting and extrapolation tasks [7,8,9] and showed best results for non-regular sampling and quarter sampling in [4,6].…”

Section: Introductionmentioning

confidence: 99%

Iterative Optimization of Quarter Sampling Masks for Non-Regular Sampling Sensors

2018 25th IEEE International Conference on Image Processing (ICIP)

Seiler

Kaup

2018

Self Cite

Non-regular sampling can reduce aliasing at the expense of noise. Recently, it has been shown that non-regular sampling can be carried out using a conventional regular imaging sensor when the surface of its individual pixels is partially covered. This technique is called quarter sampling (also 1/4 sampling), since only one quarter of each pixel is sensitive to light. For this purpose, the choice of a proper sampling mask is crucial to achieve a high reconstruction quality. In the scope of this work, we present an iterative algorithm to improve an arbitrary quarter sampling mask which results in a continuous increase of the reconstruction quality. In terms of the reconstruction algorithms, we test two simple algorithms, namely, linear interpolation and nearest neighbor interpolation, as well as two more sophisticated algorithms, namely, steering kernel regression and frequency selective extrapolation. Besides PSNR gains of +0.31 dB to +0.68 dB relative to a random quarter sampling mask resulting from our optimized mask, visually noticeable enhancements are perceptible.