Hybrid SOM+k-Means clustering to improve planning, operation and management in water distribution systems

Brentan, Bruno Melo; Lima, Gustavo Meirelles; Luvizotto, Edevar; Izquierdo, Joaquín

doi:10.1016/j.envsoft.2018.02.013

Cited by 39 publications

(24 citation statements)

References 25 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In fact, the number of clusters could be overestimated if compared to the number of observations: the population in each cluster could be too low to have any statistical or physical meaning. For this reason, this technique can be coupled with a clustering algorithm to group the closest neurons, ultimately obtaining the prescribed number of clusters required for the specific application after the mapping, as described in [16,17,25].…”

Section: Coupling Som With K-meansmentioning

confidence: 99%

See 1 more Smart Citation

Impact of the Partitioning Method on Multidimensional Adaptive-Chemistry Simulations

et al. 2020

View full text Add to dashboard Cite

The large number of species included in the detailed kinetic mechanisms represents a serious challenge for numerical simulations of reactive flows, as it can lead to large CPU times, even for relatively simple systems. One possible solution to mitigate the computational cost of detailed numerical simulations, without sacrificing their accuracy, is to adopt a Sample-Partitioning Adaptive Reduced Chemistry (SPARC) approach. The first step of the aforementioned approach is the thermochemical space partitioning for the generation of locally reduced mechanisms, but this task is often challenging because of the high-dimensionality, as well as the high non-linearity associated to reacting systems. Moreover, the importance of this step in the overall approach is not negligible, as it has effects on the mechanisms’ level of chemical reduction and, consequently, on the accuracy and the computational speed-up of the adaptive simulation. In this work, two different clustering algorithms for the partitioning of the thermochemical space were evaluated by means of an adaptive CFD simulation of a 2D unsteady laminar flame of a nitrogen-diluted methane stream in air. The first one is a hybrid approach based on the coupling between the Self-Organizing Maps with K-Means (SKM), and the second one is the Local Principal Component Analysis (LPCA). Comparable results in terms of mechanism reduction (i.e., the mean number of species in the reduced mechanisms) and simulation accuracy were obtained for both the tested methods, but LPCA showed superior performances in terms of reduced mechanisms uniformity and speed-up of the adaptive simulation. Moreover, the local algorithm showed a lower sensitivity to the training dataset size in terms of the required CPU-time for convergence, thus also being optimal, with respect to SKM, for massive dataset clustering tasks.

show abstract

Section: Coupling Som With K-meansmentioning

confidence: 99%

“…In order to group the neurons with a clustering algorithm, a dissimilarity matrix ∆ containing the distances between each neuron and the n observations of the dataset must be calculated first [25].…”

Section: Coupling Som With K-meansmentioning

confidence: 99%

Impact of the Partitioning Method on Multidimensional Adaptive-Chemistry Simulations

et al. 2020

View full text Add to dashboard Cite

show abstract

“…To define the clusters, the hybrid methodology proposed by the authors in [30], using SOMs coupled with the k-means algorithm, was used. The neurons obtained using SOMs cluster the input data by their similarity.…”

Section: Clustering Pressure Signal To Improve Ann Efficiencymentioning

confidence: 99%

“…For this study, the architecture of the SOM was defined following the discussions presented in [30], based on a tradeoff between the quantization error and the training time. The topology followed a hexagonal distribution.…”

Section: Clustering Pressure Signal To Improve Ann Efficiencymentioning

confidence: 99%

Pattern Recognition and Clustering of Transient Pressure Signals for Burst Location

Manzi

Brentan

Lima

et al. 2019

Water

Self Cite

View full text Add to dashboard Cite

A large volume of the water produced for public supply is lost in the systems between sources and consumers. An important—in many cases the greatest—fraction of these losses are physical losses, mainly related to leaks and bursts in pipes and in consumer connections. Fast detection and location of bursts plays an important role in the design of operation strategies for water loss control, since this helps reduce the volume lost from the instant the event occurs until its effective repair (run time). The transient pressure signals caused by bursts contain important information about their location and magnitude, and stamp on any of these events a specific "hydraulic signature". The present work proposes and evaluates three methods to disaggregate transient signals, which are used afterwards to train artificial neural networks (ANNs) to identify burst locations and calculate the leaked flow. In addition, a clustering process is also used to group similar signals, and then train specific ANNs for each group, thus improving both the computational efficiency and the location accuracy. The proposed methods are applied to two real distribution networks, and the results show good accuracy in burst location and characterization.

show abstract

“…The use of clustering algorithms has been reported in various application fields for dimensionality reduction in stream flow time series (Zoppou et al, 2002;), wastewater treatment plants (Gibert, 2010b and Zhao et al, 2012 use hierarchical clustering; Liukkonen, 2013 use SOM), cyclone paths identification (Camargo et al, 2004), surface temperatures (Friedel, 2012), water quality in aquifers (Conti, Gibert, 2014), health status of wind turbines (Blanco et al, 2018), and baseline air pollution levels (Gómez-Losada et al, 2018). A hybrid combination SOM+k-Means Clustering was used to improve planning, operation and management of Water Distribution Systems in Brentan et al (2018b), which can be easily extended to other environmental problems, etc. Graph theory (Herrera et al, 2015;di Nardo et al, 2018) and social network theory (Campbell et al, 2016;Brentan et al, 2017aBrentan et al, ,2018a have been used to cluster a water distribution network into sectors so as to optimize these infrastructures' management.…”

Section: Applications and Referencesmentioning

confidence: 99%

Which method to use? An assessment of data mining methods in Environmental Data Science

Gibert

Izquierdo

Sànchez–Marrè

et al. 2018

Environmental Modelling & Software

View full text Add to dashboard Cite

Data Mining (DM) is a fundamental component of the Data Science process. Over recent years a huge library of DM algorithms has been developed to tackle a variety of problems in fields such as medical imaging and traffic analysis. Many DM techniques are far more flexible than more classical numerial simulation or statistical modelling approaches. These could be usefully applied to data-rich environmental problems. Certain techniques such as artificial neural networks, clustering, case-based reasoning or Bayesian networks have been applied in environmental modelling, while other methods, like support vector machines among others, have yet to be taken up on a wide scale. There is greater scope for many lesser known techniques to be applied in environmental research, with the potential to contribute to addressing some of the current open environmental challenges. However, selecting the best DM technique for a given environmental problem is not a simple decision, and there is a lack of guidelines and criteria that helps the data scientist and environmental scientists to ensure effective knowledge extraction from data. This paper provides a broad introduction to the use of DM in Data Science processes for environmental researchers. Data Science contains three main steps (pre-processing, data mining and post-processing). This paper provides a conceptualization of Environmental Systems and a conceptualization of DM methods, which are in the core step of the Data Science process. These two elements define a conceptual framework that is on the basis of a new methodology proposed for relating the characteristics of a given environmental problem with a family of Data Mining methods. The paper provides a general overview and guidelines of DM techniques to a non-expert user, who can decide with this support which is the more suitable technique to solve their problem at hand. The decision is related to the bidimensional relationship between the type of environmental system and the type of DM method. An illustrative two way table containing references for each pair Environmental System-Data Mining method is presented and discussed. Some examples of how the proposed methodology is used to support DM method selection are also presented, and challenges and future trends are identified.

show abstract

Hybrid SOM+k-Means clustering to improve planning, operation and management in water distribution systems

Cited by 39 publications

References 25 publications

Impact of the Partitioning Method on Multidimensional Adaptive-Chemistry Simulations

Impact of the Partitioning Method on Multidimensional Adaptive-Chemistry Simulations

Pattern Recognition and Clustering of Transient Pressure Signals for Burst Location

Which method to use? An assessment of data mining methods in Environmental Data Science

Contact Info

Product

Resources

About