Window Detection in Facades Using Heatmap Fusion

Li, Chuankang; Liu, Jiaxin; Zhang, Yuanqing; Zou, Shan-Chen; Fang, Yutong

doi:10.1007/s11390-020-0253-4

Cited by 19 publications

(18 citation statements)

References 44 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To compare our method with other window extraction approaches [22][23][24][25]27], we retrained and evaluated the proposed method on several datasets: eTRIMS, ECP, CMP, Graz50, and ParisArtDeco. The pixel accuracy is used as a metric in these previous studies, which can be calculated through Equation (7).…”

Section: Comparisons With Other Window Extraction Methodsmentioning

confidence: 99%

“…The distances between the clustered windows and the detected bounding boxes are treated as a loss metric. Li et al [27] regard window detection as an issue of keypoint detection and grouping. Their method detects a window as four keypoints, allowing it to deal with irregularly distributed windows and complex facades under diverse conditions.…”

Section: Related Workmentioning

confidence: 99%

“…Unlike traditional methods, deep learning methods can deal with facades without strict structures. A number of CNN-based approaches have been proposed for facade segmentation [22][23][24][25][26] and window detection [27,28]. However, these methods only regard each window as an individual component.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DeepWindows: Windows Instance Segmentation through an Improved Mask R-CNN Using Spatial Attention and Relation Modules

Sun

Malihi

et al. 2022

IJGI

View full text Add to dashboard Cite

Windows, as key components of building facades, have received increasing attention in facade parsing. Convolutional neural networks have shown promising results in window extraction. Most existing methods segment a facade into semantic categories and subsequently employ regularization based on the structure of manmade architectures. These methods merely concern the optimization of individual windows, without considering the spatial areas or relationships of windows. This paper presents a novel windows instance segmentation method based on Mask R-CNN architecture. The method features a spatial attention region proposal network and a relation module-enhanced head network. First, an attention module is introduced in the region proposal network to generate a spatial attention map, then the attention map is multiplied with the objectness scores of the classification branch. Second, for the head network, relation modules are added to model the spatial relationships between proposals. Appearance and geometric features are combined for instance recognition. Furthermore, we constructed a new window instance segmentation dataset with 1200 annotated images. With our dataset, the average precisions of our method on detection and segmentation increased from 53.1% and 53.7% to 56.4% and 56.7% compared with Mask R-CNN. A comparison with state-of-the-art methods also proves the predominance of our proposed method.

show abstract

Section: Comparisons With Other Window Extraction Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DeepWindows: Windows Instance Segmentation through an Improved Mask R-CNN Using Spatial Attention and Relation Modules

Sun

Malihi

et al. 2022

IJGI

View full text Add to dashboard Cite

show abstract

“…[27] presented a multiview architecture to collect reliable and visible clues from nearby views and used these clues to enhance the feature representation of a target view. Some other works mainly focus on window detection [28], [29] and building facade reconstruction [30]. The above methods usually require clean building facades.…”

Section: Deep Learning-based Methodsmentioning

confidence: 99%

Deep Facade Parsing with Occlusions

2022

KSII TIIS

View full text Add to dashboard Cite

Correct facade image parsing is essential to the semantic understanding of outdoor scenes. Unfortunately, there are often various occlusions in front of buildings, which fails many existing methods. In this paper, we propose an end-to-end deep network for facade parsing with occlusions. The network learns to decompose an input image into visible and invisible parts by occlusion reasoning. Then, a context aggregation module is proposed to collect nonlocal cues for semantic segmentation of the visible part. In addition, considering the regularity of man-made buildings, a repetitive pattern completion branch is designed to infer the contents in the invisible regions by referring to the visible part. Finally, the parsing map of the input facade image is generated by fusing the results of the visible and invisible results. Experiments on both synthetic and real datasets demonstrate that the proposed method outperforms state-of-the-art methods in parsing facades with occlusions. Moreover, we applied our method in applications of image inpainting and 3D semantic modeling.

show abstract

“…Besides, RNNs are usually more difficult to train. Li et al [43] propose a novel window corner detection framework, employing a ResNet [44] to learn image features and generate heatmaps, from which locations and relationships of keypoints are decoded; finally, the keypoints are grouped together into final windows. However, this method suffers from frequent cross mismatching of keypoints, as adjacent windows usually exhibit similar patterns.…”

Section: Related Workmentioning

confidence: 99%

Cross Field-Based Segmentation and Learning-Based Vectorization for Rectangular Windows

Zhuo

Tian

Fraundorfer

2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Detection and vectorization of windows from building fac ¸ades are important for building energy modeling, civil engineering, and architecture design. However, current applications still face the challenges of low accuracy and lack of automation.In this paper we propose a new two-steps workflow for window segmentation and vectorization from fac ¸ade images. First, we propose a cross field learning-based neural network architecture, which is augmented by a grid-based self-attention module for window segmentation from rectified fac ¸ade images, resulting in pixel-wise window blobs. Second, we propose a regression neural network augmented by Squeeze-and-Excitation (SE) attention blocks for window vectorization. The network takes the segmentation results together with the original fac ¸ade image as input, and directly outputs the position of window corners, resulting in vectorized window objects with improved accuracy. In order to validate the effectiveness of our method, experiments are carried out on four public fac ¸ades image datasets, with results usually yielding a higher accuracy for the final window prediction in comparison to baseline methods on four datasets in terms of IoU score, F1 score, and pixel accuracy.

show abstract

Window Detection in Facades Using Heatmap Fusion

Cited by 19 publications

References 44 publications

DeepWindows: Windows Instance Segmentation through an Improved Mask R-CNN Using Spatial Attention and Relation Modules

DeepWindows: Windows Instance Segmentation through an Improved Mask R-CNN Using Spatial Attention and Relation Modules

Deep Facade Parsing with Occlusions

Cross Field-Based Segmentation and Learning-Based Vectorization for Rectangular Windows

Contact Info

Product

Resources

About