Synthetic images generation for semantic understanding in facility management

Abstract

Purpose This study aims to introduce a new methodology for generating synthetic images for facility management purposes. The method starts by leveraging the existing 3D open-source BIM models and using them inside a graphic engine to produce a photorealistic representation of indoor spaces enriched with facility-related objects. The virtual environment creates several images by changing lighting conditions, camera poses or material. Moreover, the created images are labeled and ready to be trained in the model. Design/methodology/approach This paper focuses on the challenges characterizing object detection models to enrich digital twins with facility management-related information. The automatic detection of small objects, such as sockets, power plugs, etc., requires big, labeled data sets that are costly and time-consuming to create. This study proposes a solution based on existing 3D BIM models to produce quick and automatically labeled synthetic images. Findings The paper presents a conceptual model for creating synthetic images to increase the performance in training object detection models for facility management. The results show that virtually generated images, rather than an alternative to real images, are a powerful tool for integrating existing data sets. In other words, while a base of real images is still needed, introducing synthetic images helps augment the model’s performance and robustness in covering different types of objects. Originality/value This study introduced the first pipeline for creating synthetic images for facility management. Moreover, this paper validates this pipeline by proposing a case study where the performance of object detection models trained on real data or a combination of real and synthetic images are compared.

Systematic Literature Review of Open Infrastructure BIM

“…Another work using semantic technology to extend the IFC standard is that of Wang et al [136], which concerns the field of facilities management by means of sensors and Internet of Things systems. Other researchers, such as Rampini and Re Cecconi in 2023, proposed the use of synthetic images, alongside real ones, generated from 3D BIM models; this proposal aimed to improve the performance of training object detection models in facilities management [137]. Semantic technologies are very useful in the field of facilities management; in 2022, Chen et al [138] developed a method to automatically detect defects in the concrete of buildings by exploiting aerial images and semantic-rich BIM models.…”

Smart Detection System of Safety Hazards in Industry 5.0

“…Focusing on the domain of facilities management, the authors in [26] propose an approach to enhance the performance of object detection algorithms. Specifically, this approach involves generating synthetically labeled images by leveraging pre-existing 3D building models and inserting them into a graphic engine.…”

“…Boyong He et al [24] Maritime surveillance Ship recognition in aerial images Kai Wang et al [25] Robot scene understanding Object detection in vending machine Tremblay et al [22] Objects detection for the household environment Rampini and Re Cecconi [26] Facilities management Facility management component object detection Akar et al [27] Industry Dataset for object detection Saleh et al [28] Urban scene understanding Semantic Segmentation Sutjaritvorakul et al [29] Construction site safety management Worker detection Neuhausen et al [30] Worker detection and tracking Lee and Lee [31] Worker fall detection…”