Discovery of extreme events-related communities in contrasting groups of physical system networks

Abstract

The latent behavior of a physical system that can exhibit extreme events such as hurricanes or rainfalls, is complex. Recently, a very promising means for studying complex systems has emerged through the concept of complex networks. Networks representing relationships between individual objects usually exhibit community dynamics. Conventional community detection methods mainly focus on either mining frequent subgraphs in a network or detecting stable communities in time-varying networks. In this paper, we formulate a novel problem-detection of predictive and phase-biased communities in contrasting groups of networks, and propose an efficient and effective machine learning solution for finding such anomalous communities. We build different groups of networks corresponding to different system's phases, such as higher or low hurricane activity, discover phase-related system components as seeds to help bound the search space of community generation in each network, and use the proposed contrast-based technique to identify the changing communities across different groups. The detected anomalous communities are hypothesized (1) to play an important role in defining the target system's state(s) and (2) skill of the system's states when used collectively in the ensemble of predictive models. When tested on the two important extreme event problems-identification of tropical cyclone-related and of African Sahel rainfall-related climate indices-our algorithm demonstrated the superior performance in terms of various skill and robustness metrics, including 8-16 % accuracy increase, as well as physical interpretability of detected communities. The experimental results also show the efficiency of our algorithm on synthetic datasets.

Suitable organization forms for knowledge management to attain sustainable competitive advantage in the renewable energy industry

“…The results obtained from big data analytics can provide a stronger decision-making power, a deeper discovery power, a more optimal procedure power, and a more precise analysis power at the same time [42]. However, when employing big data analytics, big data with characteristics of volume, velocity, variety and veracity must be satisfied, and a research methodology including analytic visualizations, data mining algorithm, predictive analytic capabilities, semantic engines, and data quality and master data management must be considered [43]. In addition, overall handling procedures consisting of data collection, preprocess, statistics and analysis, and data mining must be carefully processed [42].…”

Towards Real-Time Probabilistic Risk Assessment by Sensing Disruptive Events from Streamed News Feeds

“…the occurrence of multiple operating "phases" of a system, with feedback loops existing between these phases. In [9], the authors have attempted to identified the occurrence of "anomalous" communities in such phased-based systems. Another perspective is taken by Rahwan et al [11], where they consider how "social mobilisation" could be used to enable the detection of rare events (considered in the context of the DARPA "Network Challenge" [12] and the subsequent "Tag Challenge").…”

Automatic Detection and Correction of Multi-class Classification Errors Using System Whole-part Relationships

“…For instance, the machine learning algorithms [2,3,17] have been specifically designed to overcome underdetermined problem spaces when performing classification. While they are largely successful in their test applications, their predictions do not take advantage of hierarchical characteristics such as system-subsystem relationships, nor are the prediction results checked against these characteristics for sanity.…”

“…Classification has been extensively studied in various domains, including extreme event prediction [3,4], text classification [12], and sentiment analysis [13], etc. To the best of our knowledge, this paper is the first work addressing the detection and correction of potential errors in prediction results of a multi-class classification algorithm.…”