A practical approach to acquisition and processing of free viewpoint video

Domański, Marek; Dziembowski, Adrian; Mieloch, Dawid; Luczak, Adam; Stankiewicz, Olgierd; Wegner, Krzysztof

doi:10.1109/pcs.2015.7170037

Cited by 13 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, these methods can be hardly used directly for the refinement of sequences captured by sparse multicamera systems because only a small field of view overlaps between more distant views [62]. A simple solution for such a kind of content is to refine colors of different views hierarchically, i.e., by comparing the color of neighboring views only and multiplying correction coefficients in the final step to equalize color characteristics of all views [38], [39].…”

Section: Related Workmentioning

confidence: 99%

Color Correction for Immersive Video Applications

et al. 2021

View full text Add to dashboard Cite

In this paper, the color correction method developed for immersive video systems is presented. The proposed method significantly increases the consistency of color characteristics of multiview sequences, understood both as the temporal and the inter-view consistency, what highly improves the subjective quality of the synthesized virtual views presented to the final user of the immersive video system. Moreover, the proposal allows to significantly increase the quality of the depth maps calculated for natural sequences, e.g., for views with colors inconsistent due to different lighting conditions. It enables more efficient compression of natural multiview video, as the newest encoding standards highly depend on the quality of depth maps. In order to evaluate the performance of the proposal, three experiments were conducted. In the first one, the proposal was compared to state of the art in the typical immersive video applicationcolor correction of a natural multiview sequence. In the second experiment, the performance of the proposal was tested on the Middlebury stereo dataset. In both experiments, the quality of synthesized virtual views was assessed subjectively by a group of 70 naïve viewers. The third experiment assessed the influence of color correction on the quality of estimated depth maps. All the experiments showed that the proposal significantly increases the color consistency of the multiview content. Due to the high usefulness and robustness, the proposed color correction method became the MPEG reference software for color correction. The implementation of the method is available for other researchers on the public repository.INDEX TERMS color correction, color refinement, immersive video, 3DoF+, inter-view consistency, virtual navigation.

show abstract

Section: Related Workmentioning

confidence: 99%

Color Correction for Immersive Video Applications

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In the considered scenario, the optical axes of cameras do not have to be parallel. Therefore, in order to achieve interview consistency, the depth of a point has to be defined not as the distance from the plane of the camera that acquired this point, but as the distance from the plane of the center camera of the system [43] (for the sake of comprehension: the plane of a camera is a plane that contains the sensor of the camera).…”

Section: E Cost Function Application Detailsmentioning

confidence: 99%

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

2020

View full text Add to dashboard Cite

The paper presents a new method of depth estimation, dedicated for free-viewpoint television (FTV) and virtual navigation (VN). In this method, multiple arbitrarily positioned input views are simultaneously used to produce depth maps characterized by high interview and temporal consistencies. The estimation is performed for segments and their size is used to control the trade-off between the quality of depth maps and the processing time of depth estimation. Additionally, an original technique is proposed for the improvement of temporal consistency of depth maps. This technique uses the temporal prediction of depth, thus depth is estimated for P-type depth frames. For such depth frames, temporal consistency is high, whereas estimation complexity is relatively low. Similarly, as for video coding, I-type depth frames with no temporal depth prediction are used in order to achieve robustness. Moreover, we propose a novel parallelization technique that significantly reduces the estimation time. The method is implemented in C++ software that is provided together with this paper, so other researchers may use it as a new reference for their future works. In performed experiments, MPEG methodology was used whenever possible. The provided results demonstrate the advantages over the Depth Estimation Reference Software (DERS) developed by MPEG. The fidelity of a depth map, measured by the quality of synthesized views, is higher on average by 2.6 dB. This significant quality improvement is obtained despite a significant reduction of the estimation time, on average 4.5 times. The application of the proposed temporal consistency enhancement method increases this reduction to 29 times. Moreover, the proposed parallelization results in the reduction of the estimation time up to 130 times (using 6 threads). As there is no commonly accepted measure of the consistency of depth maps, the application of compression efficiency of depth is proposed as a measure of depth consistency.

show abstract

“…This yields deterioration of the 3D-HEVC compression performance that converges to a drastically lower HEVC simulcast compression efficiency. Moreover, for the viewpoint located outside the connecting line between camera views, the interview-prediction model should be more complex than that based on simple compensation of the horizontal disparity, as it is currently implemented in 3D-HEVC reference software [9].…”

Section: D-hevc In Non-linear Large-baseline Conditionsmentioning

confidence: 99%