Color Correction for Immersive Video Applications

Abstract

In this paper, the color correction method developed for immersive video systems is presented. The proposed method significantly increases the consistency of color characteristics of multiview sequences, understood both as the temporal and the inter-view consistency, what highly improves the subjective quality of the synthesized virtual views presented to the final user of the immersive video system. Moreover, the proposal allows to significantly increase the quality of the depth maps calculated for natural sequences, e.g., for views with colors inconsistent due to different lighting conditions. It enables more efficient compression of natural multiview video, as the newest encoding standards highly depend on the quality of depth maps. In order to evaluate the performance of the proposal, three experiments were conducted. In the first one, the proposal was compared to state of the art in the typical immersive video applicationcolor correction of a natural multiview sequence. In the second experiment, the performance of the proposal was tested on the Middlebury stereo dataset. In both experiments, the quality of synthesized virtual views was assessed subjectively by a group of 70 naïve viewers. The third experiment assessed the influence of color correction on the quality of estimated depth maps. All the experiments showed that the proposal significantly increases the color consistency of the multiview content. Due to the high usefulness and robustness, the proposed color correction method became the MPEG reference software for color correction. The implementation of the method is available for other researchers on the public repository.INDEX TERMS color correction, color refinement, immersive video, 3DoF+, inter-view consistency, virtual navigation.

View Synthesis Tool for VR Immersive Video

“…As not all camera sensors are equal, small differences in color rapidly generate incoherent depth estimations or nonhomogeneous color patches during view blending (Figure 3c). Color correction is usually needed prior to the view synthesis [22,23] or during the blending step [24,25].…”

Color consistency method for cameras with unknown model

“…Более того, методы вычислительной фотографии и глубокого обучения позволяют значительно улучшить качество изображений, получаемых с использованием плоской дифракционной оптики или металинз [3 -8]. Наряду с прогрессом в задачах повышения качества изображений [9], одной из важнейших задач при обработке цвета изображений является задача обеспечения похожести или консистентности цветов на изображениях, полученных с различных камер [10,11]. Проблема обеспечения схожести цветов при различных способах регистрации или синтеза изображения важна как при обработке фото-и видеоданных с различных источников [5,11], так и при анализе данных ДЗЗ [12,13] и синтезе виртуальной реальности [5].…”