Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera

Abstract

Figure 1: Dynamic Scene View Synthesis: (Left) A dynamic scene is captured from a monocular camera from the locations V 0 to V k . Each image captures people jumping at each time step (t = 0 to t = k). (Middle) A novel view from an arbitrary location between V 0 and V 1 (denoted as an orange frame) is synthesized with the dynamic contents observed at the time t = k. The estimated depth at V k is shown in the inset. (Right) For the novel view (orange frame), we can also synthesize the dynamic content that appeared across any views in different time (traces of the foreground in each time step are shown). More results are shown in Sec. 5 and the supplementary document and video.

Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis

“…It is also addressed as "multiview synthesis" in literature. For this task, some approaches have synthesized lifelike images [21,19,28,12,43,40]. However, they heavily rely on pose supervision or 3D annotation.…”

“…To have a better understanding of the improvements of the feedback connections, we also compare our approach with two effective few-shot recognition models: Matching Net (MN) [36] and Proto-Matching Net (PMN) [37]. Note that while conducting comparisons with other view synthesis models (such as [42,34,19,28,12,43,40]) and few-shot recognition models (such as [2,16,6]) are interesting, this is not the main focus of our paper. We aim to validate that the bowtie architecture outperforms the single-task models upon which it builds.…”