Computer Vision-based Analysis of Buildings and Built Environments: A Systematic Review of Current Approaches

The extraction of features and classification of traditional dwellings plays significant roles in preserving and ensuring the sustainable development of these structures. Currently, challenges persist in subjective classification and the accuracy of feature extraction. This study focuses on traditional dwellings in Gansu Province, China, employing a novel model named Improved Swin Transformer. This model, based on the Swin Transformer and parallel grouped Convolutional Neural Networks (CNN) branches, aims to enhance the accuracy of feature extraction and classification precision. Furthermore, to validate the accuracy of feature extraction during the prediction process and foster trust in AI systems, explainability research was conducted using Grad-CAM-generated heatmaps. Initially, the Gansu Province Traditional Dwelling Dataset (GTDD) is established. On the constructed GTDD dataset, the Improved Swin Transformer attains an accuracy of 90.03% and an F1 score of 87.44%. Comparative analysis with ResNet-50, ResNeXt-50, and Swin Transformer highlights the outstanding performance of the improved model. The confusion matrix of the Improved Swin Transformer model reveals the classification results across different regions, indicating that the primary influencing factors are attributed to terrain, climate, and cultural aspects. Finally, using Grad-CAM-generated heatmaps for explaining classifications, it is observed that the Improved Swin Transformer model exhibits more accurate localization and focuses on features compared to the other three models. The model demonstrates exceptional feature extraction ability with minimal influence from the surrounding environment. Simultaneously, through the heatmaps generated by the Improved Swin Transformer for traditional residential areas in five regions of Gansu, it is evident that the model accurately extracts architectural features such as roofs, facades, materials, windows, etc. This validates the consistency of features extracted by the Improved Swin Transformer with traditional methods and enhances trust in the model and decision-making. In summary, the Improved Swin Transformer demonstrates outstanding feature extraction ability and accurate classification, providing valuable insights for the protection and style control of traditional residential areas.

Section: Research On Machine Learning For Architectural Classificationmentioning

confidence: 99%

Classification and Model Explanation of Traditional Dwellings Based on Improved Swin Transformer

Miao,

Zhang,

Piao

et al. 2024

“…Panoramic street-level imagery is now an established source of data for many applications, including in health [1], environmental [2], and urban [3][4][5] research. In particular, the combination of geographic information systems (GIS) with SVI has been effective at enhancing urban analysis ranging from land use classification to property valuation, qualitative perception, and change detection [6].…”

Section: Introductionmentioning

confidence: 99%

“…Many of these early works acknowledged and often tackled issues arising from the relative inaccuracy of GPS positioning in urban settings [21,22,[29][30][31][32][33][34][35][36]39]. In parallel, advances in computer vision and deep learning, particularly around image classification and regression, object detection, and semantic segmentation, improved pipelines and unlocked new methodologies for architectural and urban analysis [5]. However, the integration of visual and geospatial data remains a challenge.…”

Section: Introductionmentioning

confidence: 99%

“…However, the integration of visual and geospatial data remains a challenge. On one hand, datasets are siloed and lack interoperability [5]; on the other hand, models and algorithms that are directly imported from computer vision lack any explicit use of the contextual information surrounding geotagged visual data [40]. Moreover, the unique perspective provided by SVI comes with specific limitations such as occlusions, privacy blurring, and geometric distortions [3], hampering not only visual analysis but also integration with other data sources.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust Building Identification from Street Views Using Deep Convolutional Neural Networks

Roussel,

Jacoby,

Asadipour

2024

Street view imagery (SVI) is a rich source of information for architectural and urban analysis using computer vision techniques, but its integration with other building-level data sources requires an additional step of visual building identification. This step is particularly challenging in architecturally homogeneous, dense residential streets featuring narrow buildings, due to a combination of SVI geolocation errors and occlusions that significantly increase the risk of confusing a building with its neighboring buildings. This paper introduces a robust deep learning-based method to identify buildings across multiple street views taken at different angles and times, using global optimization to correct the position and orientation of street view panoramas relative to their surrounding building footprints. Evaluating the method on a dataset of 2000 street views shows that its identification accuracy (88%) outperforms previous deep learning-based methods (79%), while methods solely relying on geometric parameters correctly show the intended building less than 50% of the time. These results indicate that previous identification methods lack robustness to panorama pose errors when buildings are narrow, densely packed, and subject to occlusions, while collecting multiple views per building can be leveraged to increase the robustness of visual identification by ensuring that building views are consistent.

“…The convergence of these technologies illustrates the increasing versatility and sophistication of imaging tools in the field of light and radiation measurement. The integration of digital technology and artificial intelligence, including computer vision, is becoming a significant factor in built-environment research, most especially in diagnostics, which is being bolstered by image processing approaches [15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Ultraviolet Radiation Transmission in Buildings’ Fenestration: Part I, Detection Methods and Approaches Using Spectrophotometer and Radiometer

Onatayo

Srinivasan

Shah³

2023

Fenestration, comprising windows and other openings in a building, influences occupant health and well-being while also enhancing energy efficiency through optimized design and placement. Selecting glazing materials that block or filter harmful UV radiation is crucial, as is having reliable methods to measure their UV transmission. This research paper, which comprises Part I of II, conducts an exhaustive analysis of the predominant methodologies and associated challenges inherent in assessing ultraviolet (UV) radiation transmission and deterioration status in fenestration, with particular emphasis on the usage of spectrophotometers and radiometers. It details the economic and operational challenges associated with these instruments and the limitations they pose in terms of sample size and the impact of glazing material degradation over time. Additionally, the paper provides a comparative analysis of UV radiation transmission characteristics across different glazing configurations such as green or yellow patterned glass, laminate glass, clear glass treated with UV protection film, and float glass. The review identifies laminated glass material as offering the best protection. Although spectrophotometers and radiometers offer high levels of accuracy in transmission data detection, their adoption is hindered by considerable financial and operational challenges. The paper underscores the need for developing alternative methodologies that are economically viable, operationally less complex, and are capable of overcoming the limitations of the traditional methods of detection, which will facilitate optimal fenestration configurations for UV protection and energy efficiency in buildings. The proposed Part II paper will explore UV imaging, image processing, and computer vision techniques as potential alternative approaches.