Reconstruction of videos taken by a non-regular sampling sensor

Jonscher, Markus; Seiler, Jurgen; Bätz, Michel; Richter, Thomas; Schnurrer, Wolfgang; Kaup, André

doi:10.1109/vcip.2015.7457801

Cited by 4 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the reconstruction of a non-regularly sampled video, however, it is obviously advantageous to make use of the temporal dependencies between neighboring frames. Since the performance of FSR highly depends on the number of available sampling points, a multi-frame FSR approach (FSR-MF) has been proposed in [7] in order to increase the reconstruction quality. It projects additional pixel information from neighboring frames by using a suitable motion estimation algorithm.…”

Section: B Multi-frame Reconstructionmentioning

confidence: 99%

“…it has been shown in [6] that the reconstruction quality of images captured by non-regular sampling sensors in multiview scenarios may be further enhanced by utilizing the spatial correlation between neighboring views. When dealing with videos, the temporal correlation between neighboring frames may also be exploited as it is done for instance in an existing multi-frame reconstruction approach [7], in superresolution [8], or video coding [9]. In this paper, a new recursive multi-frame reconstruction approach is proposed in order to increase the reconstruction quality compared to the existing multi-frame approach from [7].…”

Section: Introductionmentioning

confidence: 99%

“…When dealing with videos, the temporal correlation between neighboring frames may also be exploited as it is done for instance in an existing multi-frame reconstruction approach [7], in superresolution [8], or video coding [9]. In this paper, a new recursive multi-frame reconstruction approach is proposed in order to increase the reconstruction quality compared to the existing multi-frame approach from [7]. This is achieved by introducing a new reference order.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Recursive frequency selective reconstruction of non-regularly sampled video data

Jonscher¹,

Jaskolka²,

Seiler³

et al. 2016

2016 Picture Coding Symposium (PCS)

Self Cite

View full text Add to dashboard Cite

High resolution images can be acquired using a non-regular sampling sensor which consists of an underlying low resolution sensor that is covered with a non-regular sampling mask. The reconstructed high resolution image is then obtained during post-processing. Recently, it has been shown that the temporal correlation between neighboring frames can be exploited in order to enhance the reconstruction quality of non-regularly sampled video data. In this paper, a new recursive multi-frame reconstruction approach is proposed in order to further increase the reconstruction quality. By using a new reference order, previously reconstructed frames can be used for the subsequent motion estimation and a new weighting function allows for the incorporation of multiple pixels projected onto the same position. With the new recursive multi-frame approach, a visually noticeable average gain in PSNR of up to 1.13 dB with respect to a state-of-the-art single-frame reconstruction approach can be achieved. Compared to the existing multi-frame approach, a gain of 0.31 dB is possible. SSIM results show the same behavior as PSNR results. Additionally, the pre-reconstruction step of the existing multi-frame approach can be avoided and the new algorithm is, in general, capable of real-time processing.

show abstract

Section: B Multi-frame Reconstructionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recursive frequency selective reconstruction of non-regularly sampled video data

Jonscher¹,

Jaskolka²,

Seiler³

et al. 2016

2016 Picture Coding Symposium (PCS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Up to now, the reconstruction of the partially available image data that is captured by a non-regular sampling sensor has been performed by the two-dimensional frequency selective extrapolation (2D-FSE) [19] or the enhanced two-dimensional frequency selective reconstruction (2D-FSR) in [15]. Recently, it has been shown in [4], [20] that the reconstruction quality can be enhanced by exploiting spatial or temporal correlations between neighboring views or frames. A further enhancement of the reconstruction quality can be expected by the use of algorithms that directly reconstruct volumes [21], [22], [23].…”

Section: A Static Samplingmentioning

confidence: 99%

Dynamic Non-Regular Sampling Sensor Using Frequency Selective Reconstruction

Jonscher,

Seiler,

Lanz

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

“…Besides still images, the acquisition of video data is of great importance. In combination with quarter sampling, video acquisi- tion has been investigated for fixed quarter sampling masks [14,15] as well as for dynamic quarter sampling masks [16]. For the latter, the sampling mask changes from frame to frame and a sophisticated read-out strategy is applied such that each pixel in a 2×2 block is read exactly once within four frames.…”

Section: Introductionmentioning

confidence: 99%

Novel Consistency Check For Fast Recursive Reconstruction Of Non-Regularly Sampled Video Data

Grosche¹,

Seiler²,

Kaup³

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Quarter sampling is a novel sensor design that allows for an acquisition of higher resolution images without increasing the number of pixels. When being used for video data, one out of four pixels is measured in each frame. Effectively, this leads to a non-regular spatio-temporal sub-sampling. Compared to purely spatial or temporal sub-sampling, this allows for an increased reconstruction quality, as aliasing artifacts can be reduced. For the fast reconstruction of such sensor data with a fixed mask, recursive variant of frequency selective reconstruction (FSR) was proposed. Here, pixels measured in previous frames are projected into the current frame to support its reconstruction. In doing so, the motion between the frames is computed using template matching. Since some of the motion vectors may be erroneous, it is important to perform a proper consistency checking. In this paper, we propose faster consistency checking methods as well as a novel recursive FSR that uses the projected pixels different than in literature and can handle dynamic masks. Altogether, we are able to significantly increase the reconstruction quality by +1.01 dB compared to the state-of-the-art recursive reconstruction method using a fixed mask. Compared to a single frame reconstruction, an average gain of about +1.52 dB is achieved for dynamic masks. At the same time, the computational complexity of the consistency checks is reduced by a factor of 13 compared to the literature algorithm.

show abstract

Reconstruction of videos taken by a non-regular sampling sensor

Cited by 4 publications

References 8 publications

Recursive frequency selective reconstruction of non-regularly sampled video data

Recursive frequency selective reconstruction of non-regularly sampled video data

Dynamic Non-Regular Sampling Sensor Using Frequency Selective Reconstruction

Novel Consistency Check For Fast Recursive Reconstruction Of Non-Regularly Sampled Video Data

Contact Info

Product

Resources

About