Image-based bidirectional scene reprojection

Yang, Lei; Tse, Yu-Chiu; Sander, Pedro V.; Lawrence, Jason; Nehab, Diego; Hoppe, Hugues; Wilkins, Clara L.

doi:10.1145/2070752.2024184

Cited by 19 publications

(32 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Re-projective rendering Re-projective rendering has a long history in computer graphics, including image-based rendering [McMillan 1997], general camera models [Popescu et al 2009], and shading reuse for acceleration [Yang et al 2011]. Our system also uses re-projective rendering for HMDs, but faces a unique challenge: combining the appearance of the virtual world and the geometry of the physical world to strike the right balance between visual fidelity and navigation comfort.…”

Section: Real Walkingmentioning

confidence: 99%

See 1 more Smart Citation

Mapping virtual and physical reality

2016

View full text Add to dashboard Cite

Kong ‡ (a) virtual world floor plan, size 33m × 21m (b) real world floor plan, size 10m × 5.8m (c) lab setup, for the real world in (b) (d) virtual world view (e) user's HMD view Figure 1: Overview of our system. (a) and (b) are aerial views of the virtual and physical worlds, with their dimensions in meters. Typically, (a) is much larger than (b). The goal is to enable the user to walk freely in (b) while experiencing (a) using a head-mounted display (HMD). Our system first computes a planar map between (a) and (b), with the mapped user walking paths overlaid in green gradients. (c) is a photograph of the lab setup, including the equipment and surroundings. Our system then renders the virtual world appearance (d) for the user's HMD view (e) in a way that is compatible with the real world geometry, so that the user can faithfully see the former and comfortably navigate the latter. Note that even though the user's view is entirely blocked by the HMD, our system guides the user away from boundaries and obstacles such as the walls and furniture.

show abstract

Section: Real Walkingmentioning

confidence: 99%

“…Both planar maps [Poranne and Lipman 2014;Fu et al 2015;Chen and Weber 2015] and projective rendering [McMillan 1997;Yang et al 2011;Popescu et al 2009] have been extensively studied in computer graphics. However, our VR method has different requirements from these prior art.…”

Section: Introductionmentioning

confidence: 99%

Mapping virtual and physical reality

2016

View full text Add to dashboard Cite

show abstract

“…These methods assume structural relationships between source and destination images that are not present in general image morphing. The same is true for view interpolation in image-based rendering (see survey in [Shum et al 2003]) and for temporal upsampling of video sequences [e.g., Kang et al 2003;Mahajan et al 2009;Yang et al 2011]. Nevertheless, for specialized image domains (such as faces or periodic textures), correspondences can be found automatically [Bichsel 1996;Beymer 1996;Covell 1996;Liu et al 2002;Yang et al 2012].…”

Section: Related Workmentioning

confidence: 99%

“…Inspired by the work of Yang et al [2011], we propose a simple and efficient backward-mapping algorithm to render intermediate frames within a morph. Our method does not involve mesh rasterization.…”

Section: Direct Pixel Evaluation For Renderingmentioning

confidence: 99%

Automating Image Morphing Using Structural Similarity on a Halfway Domain

et al. 2014

Self Cite

View full text Add to dashboard Cite

The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline, and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.

show abstract

“…The key is to find, for each pixel x, its advection source x in the previous frame given the forward flow field. Inspired by [Yang et al 2011], we use an iterative back propagation solver to find the advection source. We then compute the mean saliency weighted motion change rate by Φ(vj, f ) = 1 N…”

Section: Video Analysismentioning

confidence: 99%

audeosynth

Liao

Gong

et al. 2015

ACM Trans. Graph.

View full text Add to dashboard Cite

We introduce music-driven video montage, a media format that offers a pleasant way to browse or summarize video clips collected from various occasions, including gatherings and adventures. In music-driven video montage, the music drives the composition of the video content. According to musical movement and beats, video clips are organized to form a montage that visually reflects the experiential properties of the music. Nonetheless, it takes enormous manual work and artistic expertise to create it. In this paper, we develop a framework for automatically generating music-driven video montages. The input is a set of video clips and a piece of background music. By analyzing the music and video content, our system extracts carefully designed temporal features from the input, and casts the synthesis problem as an optimization and solves the parameters through Markov Chain Monte Carlo sampling. The output is a video montage whose visual activities are cut and synchronized with the rhythm of the music, rendering a symphony of audio-visual resonance.

show abstract

Image-based bidirectional scene reprojection

Cited by 19 publications

References 0 publications

Mapping virtual and physical reality

Mapping virtual and physical reality

Automating Image Morphing Using Structural Similarity on a Halfway Domain

audeosynth

Contact Info

Product

Resources

About