Predicting Ground-Level Scene Layout from Aerial Imagery

Zhai, Menghua; Bessinger, Zachary; Workman, Scott; Jacobs, Nathan

doi:10.1109/cvpr.2017.440

Cited by 197 publications

(276 citation statements)

References 30 publications

Supporting

Mentioning

264

Contrasting

Order By: Relevance

“…This is a more challenging task since different views have little or no overlap. To tackle this problem, Zhai et al [48] try to generate panoramic ground-level images from aerial images of the same location by using a convolutional neural network. Krishna and Ali [34] propose a X-Fork and a X-Seq GAN-based structure to address the aerial to street view image translation task using an extra semantic segmentation map.…”

Section: Related Workmentioning

confidence: 99%

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

Tang

Sebe

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

204

224

View full text Add to dashboard Cite

Cross-view image translation is challenging because it involves images with drastically different views and severe deformation. In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (Selection-GAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map. The proposed SelectionGAN explicitly utilizes the semantic information and consists of two stages. In the first stage, the condition image and the target semantic map are fed into a cycled semantic-guided generation network to produce initial coarse results. In the second stage, we refine the initial results by using a multi-channel attention selection mechanism. Moreover, uncertainty maps automatically learned from attentions are used to guide the pixel loss for better network optimization. Extensive experiments on Dayton [42], CVUSA [44] and Ego2Top [1] datasets show that our model is able to generate significantly better results than the state-of-the-art methods. The source code, data and trained models are available at https://github. com/Ha0Tang/SelectionGAN .

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

Tang

Sebe

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

204

224

View full text Add to dashboard Cite

show abstract

“…Top matches (top 1 -top 5 from left to right) Figure 10: Qualitative Results on CVUSA dataset [46] for aerial-to-ground image matching. Images with green borders are the groundtruth panoramas for the corresponding query images.…”

Section: Synthesized Ground Panoramamentioning

confidence: 99%

“…Regmi and Mubarak Shah Center for Research in Computer Vision, University of Central Florida krishna.regmi7@gmail.com, shah@crcv.ucf.edu T-SNE[27] visualization of aerial and ground image features obtained using the two-stream baseline (left) and the proposed feature fusion method (right) for 100 test images on the CVUSA dataset[46].…”

mentioning

confidence: 99%

Bridging the Domain Gap for Ground-to-Aerial Image Matching

Regmi

Shah

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

137

View full text Add to dashboard Cite

The visual entities in cross-view (e.g. ground and aerial) images exhibit drastic domain changes due to the differences in viewpoints each set of images is captured from. Existing state-of-the-art methods address the problem by learning view-invariant images descriptors. We propose a novel method for solving this task by exploiting the generative powers of conditional GANs to synthesize an aerial representation of a ground-level panorama query and use it to minimize the domain gap between the two views. The synthesized image being from the same view as the reference (target) image, helps the network to preserve important cues in aerial images following our Joint Feature Learning approach. We fuse the complementary features from a synthesized aerial image with the original groundlevel panorama features to obtain a robust query representation. In addition, we employ multi-scale feature aggregation in order to preserve image representations at different scales useful for solving this complex task. Experimental results show that our proposed approach performs significantly better than the state-of-the-art methods on the challenging CVUSA dataset in terms of top-1 and top-1% retrieval accuracies. Furthermore, we evaluate the generalization of the proposed method for urban landscapes on our newly collected cross-view localization dataset with geo-reference information.

show abstract

“…Other methods geolocalize ground level images using different techniques to match features to corresponding ground-level images database (Lin et al, 2013;Müller-Budack et al, 2018). For example, ground-level imagery can be combined with aerial imagery to accom-plish a pixel-accurate semantic segmentation of the scene, or road respectively (Zhai et al, 2017;Máttyus et al, 2016). Lefèvre et al (2017) detects changes by with siamese CNNs through comparing aerial imagery and street view panoramas warped to aerial image geometry.…”

Section: Related Workmentioning

confidence: 99%

Geocoding of trees from street addresses and street-level images

Laumer

Lang

Doorn

et al. 2020

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for > 50000 trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38 % of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale.

show abstract

Predicting Ground-Level Scene Layout from Aerial Imagery

Cited by 197 publications

References 30 publications

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

Bridging the Domain Gap for Ground-to-Aerial Image Matching

Geocoding of trees from street addresses and street-level images

Contact Info

Product

Resources

About