Towards 3DoF+ 360 Video Streaming System for Immersive Media

Jeong, Jong-Beom; Lee, Soonbin; Jang, Dongmin; Ryu, Eun‐Seok

doi:10.1109/access.2019.2942771

Cited by 15 publications

(5 citation statements)

References 21 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thanks to its simplicity and scalability in implementing multiple device configurations [35], it can be easily adapted to other animated image formats, such as WebP, recommendation methods [36], [37], and other streaming protocols. In addition, by reducing the computational load of the servers, the proposed approach can act as a privacy protection solution by utilizing effective encryption methods [38]- [40] in three-screen TV solutions [41], [42]. Various client-based GIF generation realtime application scenarios for smartphones or set-top boxes can be considered.…”

Section: Discussionmentioning

confidence: 99%

LTC-GIF: Attracting More Clicks on Feature-length Sports Videos

Mujtaba¹,

Choi²,

Ryu³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a lightweight method to attract users and increase views of the video by presenting personalized artistic media -i.e, static thumbnails and animated GIFs. This method analyzes lightweight thumbnail containers (LTC) using computational resources of the client device to recognize personalized events from full-length sports videos. In addition, instead of processing the entire video, small video segments are processed to generate artistic media. This makes the proposed approach more computationally efficient compared to the baseline approaches that create artistic media using the entire video. The proposed method retrieves and uses thumbnail containers and video segments, which reduces the required transmission bandwidth as well as the amount of locally stored data used during artistic media generation. When extensive experiments were conducted on the Nvidia Jetson TX2, the computational complexity of the proposed method was 3.57 times lower than that of the SoA method. In the qualitative assessment, GIFs generated using the proposed method received 1.02 higher overall ratings compared to the SoA method. To the best of our knowledge, this is the first technique that uses LTC to generate artistic media while providing lightweight and high-performance services even on resource constrained devices.

show abstract

Section: Discussionmentioning

confidence: 99%

LTC-GIF: Attracting More Clicks on Feature-length Sports Videos

Mujtaba¹,

Choi²,

Ryu³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The concept of 6DoF system architecture contains pre-processing and post-processing modules for removing inter-view redundancy. Several studies implemented 6DoF systems by down-sampling multiple videos and eliminating the correlation among [4], [12]. Based on the proposed responses, MPEG-I proposed TMIV as a reference software for 6DoF video compression.…”

Section: A MIV Standardizationmentioning

confidence: 99%

“…The MIV standard enables video codecs to handle multiple inputs through the inter-view redundancy removal process. With the development of the MIV standard, a test model for immersive video (TMIV) has been implemented as a reference software to comply with the MIV specification [16], [17]. The TMIV encoder extracts the patches from the input views and aggregates them for generating atlases.…”

Section: A MIV Standardizationmentioning

confidence: 99%

Group-Based Adaptive Rendering System for 6DoF Immersive Video Streaming

2022

Self Cite

View full text Add to dashboard Cite

The Moving Picture Experts Group (MPEG) has started an immersive media standard project to enable multi-view video and depth representation in three-dimensional (3D) scenes. The MPEG Immersive Video (MIV) standard technology is intended to provide a limited 6 degrees of freedom (DoF) based on depth map-based image rendering (DIBR). The 6DoF immersive video system is still challenging because multiple high-quality video streams require high bandwidth and computing resources. This paper proposes a group-based adaptive rendering method for 6DoF immersive video streaming. With group-based MIV, each group can be transmitted independently, which enables adaptive transmission depending on the user's viewport. The proposed method derives weights from groups for view synthesis and allocates highquality bitstreams according to a given viewport. This paper also discussed the results of the group-based approach in the MIV, and the advantages and drawbacks of this approach are detailed. In addition, pixel rate constraint analysis has been introduced to facilitate deployment with existing video codecs. On end-to-end evaluation metrics with TMIV anchor, the proposed method saves average 37.26% Bjontegaard-delta rate (BD-rate) on the peak signal-to-noise ratio (PSNR).

show abstract

“…The previous our mobile VR streaming projects contribute 360 video streaming can be approached in a limited performance of mobile VR in [13][14][15][16][17]. Additionally, we conducted studies regarding native VR [1][2][3]. These experiences enabled us to conduct an improved study on 360 video tile-based streaming for Figure 1: Conceptual architecture of proposed system a native VR system connected to a PC based on the advantages of HEVC coding.…”

Section: Tile-based Streaming Of 360 Videosmentioning

confidence: 99%

“…Certain standards, such as 3DoF, 3DoF+, and 6DoF, have been developed, typically for enhancing user experiences with regard to activities such as moving one's head, turning, and walking in a VR environment. Few studies [1,2] improved 3DoF+ and 6DoF to reduce video quality by employing various methods such as size reduction and the elimination of correlations between videos [3].…”

Section: Introductionmentioning

confidence: 99%

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

Le¹,

Jeong²,

Lee³

et al. 2021

Computers, Materials &Amp; Continua

Self Cite

View full text Add to dashboard Cite

Recent advances in 360 video streaming technologies have enhanced the immersive experience of video streaming services. Particularly, there is immense potential for the application of 360 video encoding formats to achieve highly immersive virtual reality (VR) systems. However, 360 video streaming requires considerable bandwidth, and its performance depends on several factors. Consequently, the optimization of 360 video bitstreams according to viewport texture is crucial. Therefore, we propose an adaptive solution for VR systems using viewport-dependent tiled 360 video streaming. To increase the degrees of freedom of users, the moving picture experts group (MPEG) recently defined three degrees plus of freedom (3DoF+) and six degrees of freedom (6DoF) to support free user movement within camera-captured scenes. The proposed method supports 6DoF to allow users to move their heads freely. Herein, we propose viewport-dependent tiled 360 video streaming based on users' head movements. The proposed system generates an adaptive bitstream using tile sets that are selected according to a parameter set of user's viewport area. This extracted bitstream is then transmitted to the user's computer. After decoding, the user's viewport is generated and rendered on VR head-mounted display (HMD). Furthermore, we introduce certain approaches to reduce the motion-to-photon latency. The experimental results demonstrated that, in contrast with non-tiled streaming, the proposed method achieved high-performance 360 video streaming for VR systems, with a 25.89% BD-rate saving for Y-PSNR and 61.16% for decoding time.

show abstract

Towards 3DoF+ 360 Video Streaming System for Immersive Media

Cited by 15 publications

References 21 publications

LTC-GIF: Attracting More Clicks on Feature-length Sports Videos

LTC-GIF: Attracting More Clicks on Feature-length Sports Videos

Group-Based Adaptive Rendering System for 6DoF Immersive Video Streaming

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

Contact Info

Product

Resources

About