UprightNet: Geometry-Aware Camera Orientation Estimation From Single Images

Xian, Wenqi; Li, Zhengqi; Snavely, Noah; Fisher, Matthew; Eisenman, Jonathan; Shechtman, Eli

doi:10.1109/iccv.2019.01007

Cited by 35 publications

(24 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such, the network fails to produce meaningful estimates for cases where the images are rotated. However, it will be easy to avoid such problems in many practical applications: it is often possible to observe the gravity direction through other sensors or to pre-rotate the image based on geometric cues [78].…”

Section: Discussionmentioning

confidence: 99%

Single-Image Depth Prediction Makes Feature Matching Easier

Toft¹,

Turmukhambetov²,

Sattler³

et al. 2020

Preprint

View full text Add to dashboard Cite

Good local features improve the robustness of many 3D relocalization and multi-view reconstruction pipelines. The problem is that viewing angle and distance severely impact the recognizability of a local feature. Attempts to improve appearance invariance by choosing better local feature points or by leveraging outside information, have come with pre-requisites that made some of them impractical. In this paper, we propose a surprisingly effective enhancement to local feature extraction, which improves matching. We show that CNN-based depths inferred from single RGB images are quite helpful, despite their flaws. They allow us to pre-warp images and rectify perspective distortions, to significantly enhance SIFT and BRISK features, enabling more good matches, even when cameras are looking at the same scene but in opposite directions.

show abstract

Section: Discussionmentioning

confidence: 99%

Single-Image Depth Prediction Makes Feature Matching Easier

Toft¹,

Turmukhambetov²,

Sattler³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Calibration methods for only extrinsic parameters have been proposed that are aimed at narrow view cameras [19,32,38,39,44,45] and panoramic 360 • images [10]. These methods cannot calibrate intrinsic parameters, that is, they cannot remove distortion.…”

Section: Related Workmentioning

confidence: 99%

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

Wakai¹,

Sato²,

Ishii³

et al. 2021

Preprint

View full text Add to dashboard Cite

Although recent learning-based calibration methods can predict extrinsic and intrinsic camera parameters from a single image, the accuracy of these methods is degraded in fisheye images. This degradation is caused by mismatching between the actual projection and expected projection. To address this problem, we propose a generic camera model that has the potential to address various types of distortion. Our generic camera model is utilized for learning-based methods through a closed-form numerical calculation of the camera projection. Simultaneously to recover rotation and fisheye distortion, we propose a learning-based calibration method that uses the camera model. Furthermore, we propose a loss function that alleviates the bias of the magnitude of errors for four extrinsic and intrinsic camera parameters. Extensive experiments demonstrated that our proposed method outperformed conventional methods on two largescale datasets and images captured by off-the-shelf fisheye cameras. Moreover, we are the first researchers to analyze the performance of learning-based methods using various types of projection for off-the-shelf cameras.

show abstract

“…Visual cues such as vanishing points in indoor scene images [20,24] can be leveraged to estimate the gravity from images without external sensors such as an IMU. In addition, learning-based methods have employed visual semantics to predict gravity [9,26,35]. This gravity estimate is, in turn, beneficial to recognize visual semantics, e.g., single view depth prediction [8,29].…”

Section: Related Workmentioning

confidence: 99%

“…Notably, Zeng et al [37] and Liao et al [22] proposed to use a L 1 measure on unit vectors and spherical regression loss, respectively, to overcome the limitations of L 2 loss. More recently, UprightNet [35] and VPLNet [32] employed an angular loss (AL), and showed its high effectiveness in both gravity and surface normal predictions, respectively. In this work, we propose a new angular loss called the truncated angular loss that increases robustness to outliers in the training data.…”

Section: Related Workmentioning

confidence: 99%

Surface Normal Estimation of Tilted Images via Spatial Rectifier

Vuong

Roumeliotis

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, we present a spatial rectifier to estimate surface normals of tilted images. Tilted images are of particular interest as more visual data are captured by arbitrarily oriented sensors such as body-/robot-mounted cameras. Existing approaches exhibit bounded performance on predicting surface normals because they were trained using gravity-aligned images. Our two main hypotheses are: (1) visual scene layout is indicative of the gravity direction; and ( 2) not all surfaces are equally represented by a learned estimator due to the structured distribution of the training data, thus, there exists a transformation for each tilted image that is more responsive to the learned estimator than others. We design a spatial rectifier that is learned to transform the surface normal distribution of a tilted image to the rectified one that matches the gravity-aligned training data distribution. Along with the spatial rectifier, we propose a novel truncated angular loss that offers a stronger gradient at smaller angular errors and robustness to outliers. The resulting estimator outperforms the state-of-the-art methods including data augmentation baselines not only on ScanNet and NYUv2 but also on a new dataset called Tilt-RGBD that includes considerable roll and pitch camera motion.

show abstract

UprightNet: Geometry-Aware Camera Orientation Estimation From Single Images

Cited by 35 publications

References 48 publications

Single-Image Depth Prediction Makes Feature Matching Easier

Single-Image Depth Prediction Makes Feature Matching Easier

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

Surface Normal Estimation of Tilted Images via Spatial Rectifier

Contact Info

Product

Resources

About