Distributed data mining patterns and services: an architecture and experiments

Abstract

SUMMARY Distributed data mining implements techniques for analyzing data on distributed computing systems by exploiting data distribution and parallel algorithms. The grid is a computing infrastructure for implementing distributed high‐performance applications and solving complex problems, offering effective support to the implementation and use of data mining and knowledge discovery systems. The Web Services Resource Framework has become the standard for the implementation of grid services and applications, and it can be exploited for developing high‐level services for distributed data mining applications. This paper describes how distributed data mining patterns, such as collective learning, ensemble learning, and meta‐learning models, can be implemented as Web Services Resource Framework mining services by exploiting the grid infrastructure. The goal of this work was to design a distributed architectural model that can be exploited for different distributed mining patterns deployed as grid services for the analysis of dispersed data sources. In order to validate such an approach, we presented also the implementation of two clustering algorithms on the developed architecture. In particular, the distributed k‐means and distributed expectation maximization were exploited as pilot examples to show the suitability of the implemented service‐oriented framework. An extensive evaluation of its performance was provided. Copyright © 2011 John Wiley & Sons, Ltd.

Efficient and scalable execution of smart city parallel applications

“…The validity of this assumption has been confirmed in several studies (ie, other works 22,23 ), which experimentally show that the time needed to send local or summarized information is negligible with respect to the computation time in a large subset of distributed applications. The validity of this assumption has been confirmed in several studies (ie, other works 22,23 ), which experimentally show that the time needed to send local or summarized information is negligible with respect to the computation time in a large subset of distributed applications.…”

“…The assumption that the communication time is very low with respect to the computation time, and thus can be ignored, is based on the fact that each node sends only summary data to the adjacent nodes. The validity of this assumption has been confirmed in several studies (ie, other works 22,23 ), which experimentally show that the time needed to send local or summarized information is negligible with respect to the computation time in a large subset of distributed applications. The Petri net formalism, however, can also be used to model a more general scenario in which communication times should be explicitly taken into account.…”

A Cloud-Based Prediction Framework for Analyzing Business Process Performances

“…This section is meant to briefly illustrate the service-oriented architecture proposed in [4] for carrying out Distributed Data Mining (DDM) tasks over a Grid.…”

“…Moreover, we replace the traditional (hard) clustering of the logics-based predictive clustering framework [3] used in [7,8] with a probabilistic clustering scheme, in order to reduce the risk of obtaining lowly accurate cluster predictors (due to the greedy clustering algorithm and to the underlying approximated representation of the log). In order to overcome the severe scalability limitations of [7,8] and make our approach suitable for large logs, both the computation of (probability-aware) trace clusters and of the clusters' predictors are implemented in a parallel and distributed manner, according to the Grid-services-based conceptual architecture defined in [4] for the specification and execution of Distributed Data Mining (DDM) tasks. The underlying grid services were developed according to the WSRF (Web Services Resource Framework) specifications of the WS-Core (a component of the Globus Toolkit 4 (GT4) [9]), and deployed onto a private Cloud-computing platform.…”

“…Different distributed versions of EM algorithm have been presented in the literature. Our approach essentially exploits the distributed computation scheme defined in [4], in accordance with the DDM framework of Figure 1.…”