A Survey on 360° Images and Videos in Mixed Reality: Algorithms and Applications

Zhang, Fang-Lue; Zhao, Junhong; Zhang, Yun; Zollmann, Stefanie

doi:10.1007/s11390-023-3210-1

Cited by 3 publications

(1 citation statement)

References 120 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When it comes to cameras, there have also been developments in both software and hardware domains, enabling us to capture higher resolutions with greater fidelity as well as different formats and all possible fields of views. One of those developments have been 360º cameras where fisheye optics are used in conjunction with multiple sensors, resulting in realtime panoramas [8], previously requiring extra stitching work in order to be produced and also being less suitable to capture visual material when there are moving elements in the target area. Multiple sensor fisheye arrangements also make it possible for 3D-360 stereoscopic results for use in immersive media.…”

Section: Introductionmentioning

confidence: 99%

Visualisation of Fossilised Tree Trunks Using Immersive 3D-360 Audio-Visual and Geospatial Digitisation Techniques in XR Derived from UAS and Terrestrial Data, Aided by Machine Learning Computational Photography

Psarros,

Zouros,

Soulakellis

2023

Preprint

View full text Add to dashboard Cite

The aim of this research is to investigate and use a variety of immersive multisensory media techniques in order to create convincing digital models of fossilised tree trunks for use in XR. This is made possible via the use of geospatial data derived from sources such as aerial imaging using UAS, terrestrial material using cameras and also include both the visual and audio element for better immersion, accessible and explorable in 6 Degrees of Freedom (6DoF). Immersiveness is a key factor in order to result in output that is especially engaging to the user. Both conventional and alternative methods are explored and compared, emphasising in the advantages made possible via the help of Machine Learning Computational Photography. Material is collected using both UAS and terrestrial camera devices, including a 3D-360º camera with 6 sensors, using stitched panoramas as sources for photogrammetry processing. Difficulties such as capturing large free standing objects using terrestrial means were overcome using practical solutions involving mounts and remote streaming solutions. Conclusions indicated that superior fidelity can be achieved by the help of Machine Learning Computational Photography processes and higher resolutions and technical specs of equipment not necessarily translating to superior outputs.

show abstract

Section: Introductionmentioning

confidence: 99%

Visualisation of Fossilised Tree Trunks Using Immersive 3D-360 Audio-Visual and Geospatial Digitisation Techniques in XR Derived from UAS and Terrestrial Data, Aided by Machine Learning Computational Photography

Psarros,

Zouros,

Soulakellis

2023

Preprint

View full text Add to dashboard Cite

show abstract

SGformer: Boosting transformers for indoor lighting estimation from a single image

Zhao,

Xue,

Zhang

2024

Comp. Visual Media

View full text Add to dashboard Cite

Estimating lighting from standard images can effectively circumvent the need for resource-intensive high-dynamic-range (HDR) lighting acquisition. However, this task is often ill-posed and challenging, particularly for indoor scenes, due to the intricacy and ambiguity inherent in various indoor illumination sources. We propose an innovative transformer-based method called SGformer for lighting estimation through modeling spherical Gaussian (SG) distributions—a compact yet expressive lighting representation. Diverging from previous approaches, we explore underlying local and global dependencies in lighting features, which are crucial for reliable lighting estimation. Additionally, we investigate the structural relationships spanning various resolutions of SG distributions, ranging from sparse to dense, aiming to enhance structural consistency and curtail potential stochastic noise stemming from independent SG component regressions. By harnessing the synergy of local-global lighting representation learning and incorporating consistency constraints from various SG resolutions, the proposed method yields more accurate lighting estimates, allowing for more realistic lighting effects in object relighting and composition. Our code and model implementing our work can be found at https://github.com/junhong-jennifer-zhao/SGformer.

show abstract

Hierarchical mussel farm reconstruction from video with object tracking

Zhao,

Xue,

Vennel

et al. 2024

Journal of the Royal Society of New Zealand

View full text Add to dashboard Cite

A Survey on 360° Images and Videos in Mixed Reality: Algorithms and Applications

Cited by 3 publications

References 120 publications

Visualisation of Fossilised Tree Trunks Using Immersive 3D-360 Audio-Visual and Geospatial Digitisation Techniques in XR Derived from UAS and Terrestrial Data, Aided by Machine Learning Computational Photography

Visualisation of Fossilised Tree Trunks Using Immersive 3D-360 Audio-Visual and Geospatial Digitisation Techniques in XR Derived from UAS and Terrestrial Data, Aided by Machine Learning Computational Photography

SGformer: Boosting transformers for indoor lighting estimation from a single image

Hierarchical mussel farm reconstruction from video with object tracking

Contact Info

Product

Resources

About