VR content creation and exploration with deep learning: A survey

Wang, Miao; Lyu, Xu-Quan; Li, Yijun; Zhang, Fang-Lue

doi:10.1007/s41095-020-0162-z

Cited by 63 publications

(36 citation statements)

References 142 publications

(165 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As image derived from old times and from Internets is always with low quality, for the Li culture reservations, a novel deep learning convolutional neural network-(CNN-) based virtual reality framework is introduced in this study. Blue image deconvolution is an ill-posed problem [10][11][12][13] addressed by the regularization methods.…”

Section: Blurred Li-style Boat-shaped House Restorationmentioning

confidence: 99%

Research on Convolutional Neural Network-Based Virtual Reality Platform Framework for the Intangible Cultural Heritage Conservation of China Hainan Li Nationality: Boat-Shaped House as an Example

Deng

Kim

Shen

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Hainan is located at the southernmost tip of China, since ancient times it has always occupied an important position on the Silk Road. Hainan culture is dominated by minority and marine cultures and has a rich intangible cultural heritage. Hainan has always been committed to the development and utilization of its wide cultural heritage, and the development direction is mainly based on live display and folk activities. In May 2020, the Chinese government announced the establishment of the Hainan Free Trade Port Policy and System, the establishment of a Hainan International Free Trade Zone, and the development of tourism, modern services, and high-tech industry. All these put forward higher requirements for the protection of Hainan’s cultural heritage, not just traditional ways to protect and promote, but also to use the dividends of current scientific and technological development to keep up with the times to protect and promote. The integration of digital technology will be the development direction of cultural heritage and intangible cultural heritage. This paper enumerates and analyzes other cases and academic directions of intangible cultural heritage, combined with the present situation of intangible cultural heritage in Hainan. It also analyzes the predicament of handiwork inheritance in Hainan intangible cultural heritage, expounds the structure, humanistic connotation, and construction skills of Li nationality ship house, and summarizes the role of a novel deep learning convolutional neural network- (CNN-) based virtual reality framework of intangible cultural heritage conservation in promoting the intangible cultural heritage of traditional skills. It also puts forward the scheme and heritage conservation virtual reality content construction and provides the process of building a virtual reality platform for the intangible cultural heritage of ship-shaped houses, which as an example can be used as a reference for intangible cultural heritage researchers in other areas. At the same time, it fills the gap for the artificial intelligence-based digitization of the intangible cultural heritage.

show abstract

Section: Blurred Li-style Boat-shaped House Restorationmentioning

confidence: 99%

Research on Convolutional Neural Network-Based Virtual Reality Platform Framework for the Intangible Cultural Heritage Conservation of China Hainan Li Nationality: Boat-Shaped House as an Example

Deng

Kim

Shen

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…In recent years, some researchers [5,7,11,13,14,19,26,29,31,32] have used Generative Adversarial Networks (GANs) for high-quality image-to-image translation. GAN-based methods can generate high-quality artistic works that can be considered as real.…”

Section: Introductionmentioning

confidence: 99%

Arbitrary Style Transfer via Multi-Adaptation Network

Deng

Tang

Dong

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

136

122

View full text Add to dashboard Cite

Arbitrary style transfer is a significant topic with research value and application prospect. A desired style transfer, given a content image and referenced style painting, would render the content image with the color tone and vivid stroke patterns of the style painting while synchronously maintaining the detailed content structure information. Style transfer approaches would initially learn content and style representations of the content and style references and then generate the stylized images guided by these representations. In this paper, we propose the multi-adaptation network which involves two self-adaptation (SA) modules and one co-adaptation (CA) module: the SA modules adaptively disentangle the content and style representations, i.e., content SA module uses position-wise self-attention to enhance content representation and style SA module uses channel-wise self-attention to enhance style representation; the CA module rearranges the distribution of style representation based on content representation distribution by calculating the local similarity between the disentangled content and style features in a non-local fashion. Moreover, a new disentanglement loss function enables our network to extract main style patterns and exact content structures to adapt to various input images, respectively. Various qualitative and quantitative experiments demonstrate that the proposed multi-adaptation network leads to better results than the state-of-the-art style transfer methods.

show abstract

“…When describing the panoramic video projection, the first step is to decompose it into a single frame image for processing. Taking the principle of geometric projection as an example [14], for a spherical panoramic image, firstly, it is mapped to different geometry models according to different projection formats; then, the geometric patches are expanded [15], and the patches are rearranged according to different layout methods, and finally a conventional plane rectangular image is formed, then the image is the two-dimensional plane frame corresponding to the panoramic frame.…”

Section: B Vr Video Projection Format Methodsmentioning

confidence: 99%

VR Technology in Bluish- white Porcelain Display Design of Fanchang Kiln

Hu¹,

Zhang

2024

IEEE Access

View full text Add to dashboard Cite

With the world entering the new millennium, many rare cultural relics and architectural sites are in danger. It is very urgent and important to carry out rescue protection measures for some precious architectural heritages. This paper mainly studies the application of VR technology in the Bluish-white porcelain display design of Fanchang kiln. This paper analyzes the display elements of Bluish-white porcelain of Fanchang kiln from the principles and characteristics of Bluish-white porcelain display of Fanchang kiln, and discusses the characteristics of virtual display of Bluish-white porcelain of Fanchang kiln based on the characteristics of virtual display. The Bluish-white porcelain of Fanchang kiln is classified according to the use function, and the virtual display characteristics of different types of Fanchang kiln Bluish-white porcelain and their suitable virtual display mode are explored. Finally, combined with the panoramic image display technology and graphic VR display technology, the best virtual display mode suitable for different types of Fanchang kiln Bluish-white porcelain is made, and the interactive design is realized in the virtual software, and then the virtual display application is completed in the interactive projection of network and fixed occasions. Through the exploration of the virtual display mode of Fanchang kiln Bluish-white porcelain, the best virtual expression mode suitable for different types of Fanchang kiln Bluish-white porcelain is found. With the support of virtual reality technology, it realizes all-round browsing, scene roaming and information interaction of Fanchang kiln Bluish-white porcelain, providing an interactive platform for designers and consumers, so that the design can better meet the market demand, Improve efficiency. Experimental results show that the system runs smoothly in the application, and the user experience reaches more than 93.2%, which meets the actual application requirements.

show abstract

VR content creation and exploration with deep learning: A survey

Cited by 63 publications

References 142 publications

Research on Convolutional Neural Network-Based Virtual Reality Platform Framework for the Intangible Cultural Heritage Conservation of China Hainan Li Nationality: Boat-Shaped House as an Example

Research on Convolutional Neural Network-Based Virtual Reality Platform Framework for the Intangible Cultural Heritage Conservation of China Hainan Li Nationality: Boat-Shaped House as an Example

Arbitrary Style Transfer via Multi-Adaptation Network

VR Technology in Bluish- white Porcelain Display Design of Fanchang Kiln

Contact Info

Product

Resources

About