Deep Facade Parsing with Occlusions

Abstract

Correct facade image parsing is essential to the semantic understanding of outdoor scenes. Unfortunately, there are often various occlusions in front of buildings, which fails many existing methods. In this paper, we propose an end-to-end deep network for facade parsing with occlusions. The network learns to decompose an input image into visible and invisible parts by occlusion reasoning. Then, a context aggregation module is proposed to collect nonlocal cues for semantic segmentation of the visible part. In addition, considering the regularity of man-made buildings, a repetitive pattern completion branch is designed to infer the contents in the invisible regions by referring to the visible part. Finally, the parsing map of the input facade image is generated by fusing the results of the visible and invisible results. Experiments on both synthetic and real datasets demonstrate that the proposed method outperforms state-of-the-art methods in parsing facades with occlusions. Moreover, we applied our method in applications of image inpainting and 3D semantic modeling.

LCA Calculation of Retrofitting Scenarios Using Geometric Model Reconstruction and Semantic Enrichment of Point Clouds and Images

“…Liu et al introduce a novel translational symmetrybased approach to façade parsing in order to geometrically reconstruct buildings (Liu et al 2022), further refining their previous DeepFacade approach (Liu et al 2017). Also, Ma et al improve the pixel-wise classification of DeepFacade by proposing an end-to-end deep network for façade parsing using occlusion reasoning (Ma et al 2022). Also, Zhang et al improved the accuracy of their deep learning approach for detecting building façade elements compared to DeepFacade by considering prior knowledge (Zhang et al 2022).…”