A 360-degree Panoramic Image Inpainting Network Using a Cube Map

Abstract

Inpainting has been continuously studied in the field of computer vision. As artificial intelligence technology developed, deep learning technology was introduced in inpainting research, helping to improve performance. Currently, the input target of an inpainting algorithm using deep learning has been studied from a single image to a video. However, deep learning-based inpainting technology for panoramic images has not been actively studied. We propose a 360-degree panoramic image inpainting method using generative adversarial networks (GANs). The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format, which has relatively little distortion and uses it as a training network. Since the cube map format is used, the correlation of the six sides of the cube map should be considered. Therefore, all faces of the cube map are used as input for the whole discriminative network, and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image. The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.

Viewport-Oriented Panoramic Image Inpainting

“…We compare our model with the following GAN-based baselines: the planar image inpainting models Contextual Residual Aggregation(CRA) [6] and LBAM [5], and the PI model PIINET [9]. All the baseline models are re-trained separately on 3D60 and 360-SP datasets.…”

“…Input CRA [6] LBAM [5] Original PIINET [9] Ours We also compare our model quantitatively with baselines with mask ratios from 0.2 to 0.5, as shown in Table 1. We randomly select three viewports with 90 • FOV for viewport quality evaluation.…”

“…Some of them improve the off-the-shelf methods for PIs by applying either semantic conditioning [2,8] or depth information [1]. Besides, some researchers are interested in exploring other projection formats (such as Cubemap in PIINET [9]) to introduce less distortion. PIs are usually viewed with freely changeable viewports via Head Mounted Displays (HMDs).…”

An Ultra-Fast Power Prediction Method Based on Simplified LSSVM Hyperparameters Optimization for PV Power Smoothing

“…Chaos is a widespread phenomenon in nonlinear systems, and chaotic mapping instead of traditional probability distribution is used to initialize the population which can enhance the traversal and uniformity of the population [24]. The cube map which has better uniformity is chosen to complete the initialization of the gray wolf.…”

“…where y(n) is the chaos number generated by chaos initialization and n is the size of the gray wolf population. Chaos is a complex system with unpredictable behavior, and mapping is to associate chaotic behavior with a parameter by a function [24]. The original pseudo-random numbers are replaced by chaotic numbers in the proposed algorithm, and the position is calculated.…”