Learning Dense Wide Baseline Stereo Matching for People

Abstract

Existing methods for stereo work on narrow baseline image pairs giving limited performance between wide baseline views. This paper proposes a framework to learn and estimate dense stereo for people from wide baseline image pairs. A synthetic people stereo patch dataset (S2P2) is introduced to learn wide baseline dense stereo matching for people. The proposed framework not only learns human specific features from synthetic data but also exploits pooling layer and data augmentation to adapt to real data. The network learns from the human specific stereo patches from the proposed dataset for wide-baseline stereo estimation. In addition to patch match learning, a stereo constraint is introduced in the framework to solve wide baseline stereo reconstruction of humans. Quantitative and qualitative performance evaluation against state-of-the-art methods of proposed method demonstrates improved wide baseline stereo reconstruction on challenging datasets. We show that it is possible to learn stereo matching from synthetic people dataset and improve performance on real datasets for stereo reconstruction of people from narrow and wide baseline stereo data.

Simulation-Based Estimation of the Number of Cameras Required for 3D Reconstruction in a Narrow-Baseline Multi-Camera Setup

“…By choosing this type of microscope, we subject ourselves to the limitation of the available baseline for stereo camera pairs used for reconstruction. CMO microscopes must be considered as narrow baseline systems since their inter-camera-angle (stereo angle) is typically less than [ 16 ]. While the benefits of using true multi-view camera setups for 3D reconstruction of surfaces have been investigated in previous work, this has only been done for wide-baseline camera systems with :…”

MessyTable: Instance Association in Multiple Camera Views

“…The angle is computed between line-of-sights of cameras in each image pair. A significant portion of image pairs have larger angles than that in wide-baseline stereo matching, which is typically capped at 45 • [3] an extremely diverse distribution of camera poses and large variance of angle difference between cameras, as shown in Figure 3.…”

“…We report interesting findings with some popular baselines, and discuss how this dataset could help inspire new problems and catalyse more robust formulations to tackle real-world instance association problems. 3…”

Temporal Consistency Loss for High Resolution Textured and Clothed 3DHuman Reconstruction from Monocular Video

“…There has been significant progress in 2D human pose estimation [8,2], 2D human segmentation [14,49] and 3D human pose estimation from monocular video [25,48,45] to understand the coarse geometry of the human body. Recent research has learnt to estimate full 3D human shape from a single image with impressive results [41,43,52,20,46,6]. However temporally consistent textured 3D reconstruction of clothed humans from monocular video remains a challenging problem due to the large variation in action, clothing, hair, camera viewpoint, body shape and pose.…”