An efficient <i>local</i> Algorithm for Distributed Multivariate Regression in Peer-to-Peer Networks

Bhaduri, Kanishka; Kargupta, Hillol

doi:10.1137/1.9781611972788.14

Cited by 13 publications

(13 citation statements)

References 28 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can help in performing complex data mining tasks in a decentralized and efficient fashion [2][3][4]. Various data mining algorithms have been developed to work effectively on P2P networks, such as multivariate regression [5], decision tree induction [6], eigen monitoring [7], and classification [8].…”

Section: Related Workmentioning

confidence: 99%

Peer‐to‐peer distributed text classifier learning in PADMINI

Zhu

Mahule

Dutta

et al. 2012

Statistical Analysis

Self Cite

View full text Add to dashboard Cite

Popular Internet document repositories, such as online newspapers, digital libraries, and blogs store large amount of text and image data that are frequently accessed by large number of users. Users' input through collaborative commenting or tagging can be very useful in organizing and classifying documents. Some web sites (e.g. Google Image Labeler) support a collection of tags and labels, but a large fraction of these sites do not currently support such activities. Moreover, relying upon centrally controlled web‐service providers for such support is probably not a good idea if the objective is to make the collaborative inputs publicly available. Often, business entities offering such web‐based tagging environments end up owning and monetizing the result of the collective effort. This paper takes a step toward addressing this problem—it proposes a peer‐to‐peer (P2P) system (PADMINI), powered by distributed data mining algorithms. In particular, it focuses on learning a P2P classifier from tagged text data. This paper describes the PADMINI system and the distributed text classifier learning components; text classification is posed as a linear program and an asynchronous distributed algorithm is used to solve it. It also presents extensive empirical results on text data obtained from the Hubble Space Telescope (HST) proposal abstract database. Copyright © 2012 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 2012

show abstract

Section: Related Workmentioning

confidence: 99%

Peer‐to‐peer distributed text classifier learning in PADMINI

Zhu

Mahule

Dutta

et al. 2012

Statistical Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first application of local algorithms to peer-to-peer data mining is the MajorityRule algorithm by Wolff and Schuster [1]. Since then, local algorithms were developed for other data mining tasks e.g., decision tree induction [24], multivariate regression [6], outlier detection [3], L2 norm monitoring [4], approximated sum [25], and more. The algorithm for L2 thresholding, and an initial application of that algorithm for k-means monitoring were first presented in a previous publication by the authors of this paper [4].…”

Section: Experiments With K-meansmentioning

confidence: 99%

“…These include association rule mining [1], facility location [2], outlier detection [3], L2 norm monitoring [4], classification [5], and multivariate regression [6]. In all these cases, resource consumption was shown to converge to a constant when the number of nodes is increased.…”

Section: Introductionmentioning

confidence: 99%

“…First, we generalize a common theorem underlying the local algorithms in [1], [2], [4], [5], [6] extending it from R to R d . Next, we describe a generic algorithm, relying on the said generalized theorem, which can be used to compute arbitrarily complex functions of the average of the data in a distributed system; we show how the said algorithm can be extended to other linear combinations of data, including weighted averages of selections from the data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Generic Local Algorithm for Mining Data Streams in Large Distributed Systems

Wolff

Bhaduri

Kargupta

2009

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Abstract-In a large network of computers or wireless sensors, each of the components (henceforth, peers) has some data about the global state of the system. Much of the system's functionality such as message routing, information retrieval and load sharing relies on modeling the global state. We refer to the outcome of the function (e.g., the load experienced by each peer) as the model of the system. Since the state of the system is constantly changing, it is necessary to keep the models up-to-date.Computing global data mining models e.g. decision trees, kmeans clustering in large distributed systems may be very costly due to the scale of the system and due to communication cost, which may be high. The cost further increases in a dynamic scenario when the data changes rapidly. In this paper we describe a two step approach for dealing with these costs. First, we describe a highly efficient local algorithm which can be used to monitor a wide class of data mining models. Then, we use this algorithm as a feedback loop for the monitoring of complex functions of the data such as its k-means clustering. The theoretical claims are corroborated with a thorough experimental analysis.

show abstract

“…Mining web browsing history to form P2P community based on similar interest set is proposed in [23]. A very recent paper by Bhaduri [24] proposed a local algorithm for multivariate regression in P2P network.…”

Section: Related Workmentioning

confidence: 99%

A communication efficient probabilistic algorithm for mining frequent itemsets from a peer‐to‐peer network

Datta¹,

Kargupta

2009

Statistical Analysis

Self Cite

View full text Add to dashboard Cite

Abstract:Data intensive large-scale distributed systems like peer-to-peer (P2P) networks are becoming increasingly popular where centralization of data is impossible for mining and analysis. Unfortunately, most of the existing data mining algorithms work only when data can be accessed in its entirety. Finding all the network-wide frequent itemsets is computationally difficult and usually has large communication overhead in such environment. This paper focuses on developing a communication efficient algorithm for discovering frequent itemsets from a P2P network. A sampling-based approach is adopted to find approximate solution instead of an exact solution with probabilistic guarantee. The benefit of approximation technique is reflected in the low communication overhead in discovering majority of frequent itemsets with probabilistic guarantee. The main principal followed by the algorithm assumes that an independent and identically distributed (iid) sample of the entire data is available at one location to generate a set of candidate itemsets. Collecting iid sample from a P2P network is a challenging problem because of varying degrees of connectivity and sizes of data shared. The paper first addresses this issue and shows how an iid sample of nodes and data can be collected from a P2P network using random walk. It applies the proposed sampling technique to identify most of the frequent itemsets from a P2P network. Theoretical analysis shows how to decide about optimum sample size and minimize communication to compute the results. Experimental results show that the proposed algorithm discovers all of the network-wide frequent itemsets using communication that scales sublinearly with network and datasize. 

show abstract

An efficient local Algorithm for Distributed Multivariate Regression in Peer-to-Peer Networks

Cited by 13 publications

References 28 publications

Peer‐to‐peer distributed text classifier learning in PADMINI

Peer‐to‐peer distributed text classifier learning in PADMINI

A Generic Local Algorithm for Mining Data Streams in Large Distributed Systems

A communication efficient probabilistic algorithm for mining frequent itemsets from a peer‐to‐peer network

Contact Info

Product

Resources

About