Symmetric Non-rigid Structure from Motion for Category-Specific Object Structure Estimation

Gao, Yuan; Yuille, Alan

doi:10.1007/978-3-319-46475-6_26

Cited by 17 publications

(27 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The 3D structure is specified by the 3D keypoints in Pascal3D+ [51] and the corresponding keypoints in the 2D images are from Berkeley [5]. These are the same experimental settings as used in [29,15].…”

Section: Experimental Designmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Symmetry and/or Manhattan Properties for 3D Object Structure Estimation from Single and Multiple Images

Gao

Yuille

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Many man-made objects have intrinsic symmetries and Manhattan structure. By assuming an orthographic projection model, this paper addresses the estimation of 3D structures and camera projection using symmetry and/or Manhattan structure cues, which occur when the input is singleor multiple-image from the same category, e.g., multiple different cars. Specifically, analysis on the single image case implies that Manhattan alone is sufficient to recover the camera projection, and then the 3D structure can be reconstructed uniquely exploiting symmetry. However, Manhattan structure can be difficult to observe from a single image due to occlusion. To this end, we extend to the multipleimage case which can also exploit symmetry but does not require Manhattan axes. We propose a novel rigid structure from motion method, exploiting symmetry and using multiple images from the same category as input. Experimental results on the Pascal3D+ dataset show that our method significantly outperforms baseline methods. 1 We experimented with using Manhattan for the multiple-image case, but found it gave negligible improvement. Please see the derivations of using Manhattan for the multiple-image case in the supplementary material.

show abstract

Section: Experimental Designmentioning

confidence: 99%

“…For evaluation we report the rotation error e R and the shape error e S , as in [11,12,3,17,15]. The 3D groundtruth and our estimates may have different scales, so we normalize them before evaluation.…”

Section: Experimental Designmentioning

confidence: 99%

Exploiting Symmetry and/or Manhattan Properties for 3D Object Structure Estimation from Single and Multiple Images

Gao

Yuille

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is considered to be the largest database available that contains occluded images. Note that although the occlusions may commonly occurs on the objects of interests in practice [50], [51], the Multi-PIE database does not contain images with occlusions. Thus, as a complementary experiment, we use all the 434 subjects from the Accessory category for testing, and their corresponding images from the Normal category as gallery.…”

Section: A Performance On the Insufficient Training Samples Problemmentioning

confidence: 99%

Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples

Gao

Yuille

2017

IEEE Trans. on Image Process.

Self Cite

227

View full text Add to dashboard Cite

This paper addresses the problem of face recognition when there is only few, or even only a single, labeled examples of the face that we wish to recognize. Moreover, these examples are typically corrupted by nuisance variables, both linear (i.e., additive nuisance variables, such as bad lighting and wearing of glasses) and non-linear (i.e., non-additive pixel-wise nuisance variables, such as expression changes). The small number of labeled examples means that it is hard to remove these nuisance variables between the training and testing faces to obtain good recognition performance. To address the problem, we propose a method called semi-supervised sparse representation-based classification. This is based on recent work on sparsity, where faces are represented in terms of two dictionaries: a gallery dictionary consisting of one or more examples of each person, and a variation dictionary representing linear nuisance variables (e.g., different lighting conditions and different glasses). The main idea is that: 1) we use the variation dictionary to characterize the linear nuisance variables via the sparsity framework and 2) prototype face images are estimated as a gallery dictionary via a Gaussian mixture model, with mixed labeled and unlabeled samples in a semi-supervised manner, to deal with the non-linear nuisance variations between labeled and unlabeled samples. We have done experiments with insufficient labeled samples, even when there is only a single labeled sample per person. Our results on the AR, Multi-PIE, CAS-PEAL, and LFW databases demonstrate that the proposed method is able to deliver significantly improved performance over existing methods.

show abstract

“…The most standard approach to address the inherent ambiguity of the NRSfM problem is by assuming the underlying 3D shape is low rank. In order to estimate such low rank model, factorization-based approaches have been typically used [4,10,19,31,37]. Alternatively, other approaches impose the low-rank constraint by means of robust PCAlike formulations in which the rank of a matrix representing the shape is minimized.…”

Section: Related Workmentioning

confidence: 99%

DUST: Dual Union of Spatio-Temporal Subspaces for Monocular Multiple Object 3D Reconstruction

Agudo

Moreno-Noguer

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We present an approach to reconstruct the 3D shape of multiple deforming objects from incomplete 2D trajectories acquired by a single camera. Additionally, we simultaneously provide spatial segmentation (i.e., we identify each of the objects in every frame) and temporal clustering (i.e., we split the sequence into primitive actions). This advances existing work, which only tackled the problem for one single object and non-occluded tracks. In order to handle several objects at a time from partial observations, we model point trajectories as a union of spatial and temporal subspaces, and optimize the parameters of both modalities, the nonobserved point tracks and the 3D shape via augmented Lagrange multipliers. The algorithm is fully unsupervised and results in a formulation which does not need initialization. We thoroughly validate the method on challenging scenarios with several human subjects performing different activities which involve complex motions and close interaction. We show our approach achieves state-of-the-art 3D reconstruction results, while it also provides spatial and temporal segmentation.

show abstract

Symmetric Non-rigid Structure from Motion for Category-Specific Object Structure Estimation

Cited by 17 publications

References 37 publications

Exploiting Symmetry and/or Manhattan Properties for 3D Object Structure Estimation from Single and Multiple Images

Exploiting Symmetry and/or Manhattan Properties for 3D Object Structure Estimation from Single and Multiple Images

Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples

DUST: Dual Union of Spatio-Temporal Subspaces for Monocular Multiple Object 3D Reconstruction

Contact Info

Product

Resources

About