Automatic digital twin data model generation of building energy systems from piping and instrumentation diagrams

Abstract

Buildings directly and indirectly emit a large share of current CO2 emissions. There is a high potential for CO2 savings through modern control methods in building automation systems (BAS) like model predictive control (MPC). For a proper control, MPC needs mathematical models to predict the future behavior of the controlled system. For this purpose, digital twins of the building can be used. However, with current methods in existing buildings, a digital twin set up is usually laborintensive. Especially connecting the different components of the technical system to an overall digital twin of the building is time-consuming. Piping and instrument diagrams (P&ID) can provide the needed information, but it is necessary to extract the information and provide it in a standardized format to process it further. In this work, we present an approach to recognize symbols and connections of P&ID from buildings in a completely automated way. There are various standards for graphical representation of symbols in P&ID of building energy systems. Therefore, we use different data sources and standards to generate a holistic training data set. We apply algorithms for symbol recognition, line recognition and derivation of connections to the data sets. Furthermore, the result is exported to a format that provides semantics of building energy systems. The symbol recognition, line recognition and connection recognition show good results with an average precision of 93.7%, which can be used in further processes like control generation, (distributed) model predictive control or fault detection. Nevertheless, the approach needs further research.

A review of deep learning methods for digitisation of complex documents and engineering diagrams

“…The reviewed literature is listed by application and extracted data type in Table 1. Amongst these applications, there has been a considerable research focus on P&IDs (Rahul et al 2019;Sinha et al 2019;Yu et al 2019;Mani et al 2020;Gao et al 2020;Elyan et al 2020a;Moreno-García et al 2020;Jamieson et al 2020;Nurminen et al 2020;Paliwal et al 2021a;Moon et al 2021;Kim et al 2021b;Stinner et al 2021;Paliwal et al 2021b;Toral et al 2021;Bhanbhro et al 2022;Hantach et al 2021). Another research area is architecture diagram digitisation (Ziran and Marinai 2018;Zhao et al 2020;Rezvanifar et al 2020;Kim et al 2021a;Renton et al 2021;Jakubik et al 2022).…”

“…Most of the P&ID digitisation literature focussed on the extraction of specific data types (Sinha et al 2019;Gao et al 2020;Elyan et al 2020a;Jamieson et al 2020;Nurminen et al 2020;Moon et al 2021;Kim et al 2021b;Stinner et al 2021;Paliwal et al 2021b;Toral et al 2021). There is a particular focus on P&ID symbols (Elyan et al 2020a;Nurminen et al 2020;Paliwal et al 2021b).…”

“…Other studies focussed on the text (Jamieson et al 2020;Francois et al 2022) or connectors (Moon et al 2021). Studies that presented methods for multiple element types were also seen (Gao et al 2020;Stinner et al 2021). For instance, Gao et al ( 2020) created a Region-based Fully Convolutional Network (R-FCN) (Dai et al 2016) component detection method and a SegLink (Shi et al 2017a) based text detection method.…”

“…For instance, Gao et al ( 2020) created a Region-based Fully Convolutional Network (R-FCN) (Dai et al 2016) component detection method and a SegLink (Shi et al 2017a) based text detection method. Meanwhile, Stinner et al (2021) presented work on extracting symbols, lines and line crossings, however they did not consider the text.…”

“…Faster R-CNN (Ren et al 2015) based methods were popular for engineering symbol detection (Ziran and Marinai 2018;Nguyen et al 2021;Gao et al 2020;Stinner et al 2021;Hu et al 2021;Joy and Mounsef 2021;Sarkar et al 2022;Jakubik et al 2022;Zheng et al 2022). Faster R-CNN is a two-stage object detector presented in 2015.…”