Bloom filter based optimization on HBase with MapReduce

Bhushan, Mayank; Banerjea, Shashwati; Yadav, Sumit

doi:10.1109/icdmic.2014.6954230

Cited by 9 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Probabilistic filters have been used to improve query execution performance by avoiding unnecessary disk storage access. Google BigTable [19], Apache HBase [20] and Apache Cassandra [21] are the popular Big Data solutions that utilize Bloom Filter, the most popular probabilistic filter. Several data placement [22] [23] and partitioning techniques [8] [9] [10] [11] [12] have been proposed to improve data transfer throughput among nodes in a Big Data cluster.…”

Section: Related Workmentioning

confidence: 99%

“…Eventual Consistency is a popular choice because it allows faster read/write operations, and Bloom Filter plays a vital role in improving query execution performance by avoiding disk access in the case where data do not exist on disk. Examples of popular Big Data systems incorporating these features include Google BigTable [19], Apache HBase [20] and Apache Cassandra [21]. Among these, we choose Cassandra to be our experimental Big Data system.…”

Section: Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

Improving Lookup and Query Execution Performance in Distributed Big Data Systems using Cuckoo Filter

Mosharraf

Adnan

2021

Preprint

View full text Add to dashboard Cite

Performance is a critical concern when reading and writing data from billions of records stored in Big Data warehouse. We introduce two scopes for query performance improvement. One is to improve performance of lookup queries after data deletion in Big Data systems that use Eventual Consistency. We propose a scheme to improve lookup performance after data deletion by using Cuckoo Filter. Another scope for improvement is to avoid unnecessary network round-trip for querying in remote nodes in a distributed Big Data cluster when it is known that the nodes do not have requested partition of data. We propose a scheme using probabilistic filters that are looked up before querying remote nodes, so that queries resulting in no data can be skipped from passing through the network. We evaluate our schemes with Cassandra using real dataset and show that each scheme can improve performance of lookup queries for up to 100%.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Experiments Setupmentioning

confidence: 99%

Improving Lookup and Query Execution Performance in Distributed Big Data Systems using Cuckoo Filter

Mosharraf

Adnan

2021

Preprint

View full text Add to dashboard Cite

show abstract

Spam analysis of big reviews dataset using Fuzzy Ranking Evaluation Algorithm and Hadoop

Dhingra

Yadav

2017

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

BF-MapReduce: A Bloom Filter Based Efficient Lightweight Search

Tan

Zhou

Zhang

et al. 2015

2015 IEEE Conference on Collaboration and Internet Computing (CIC)

View full text Add to dashboard Cite

MapReduce is an attractive programming model for large-scale data-parallel applications. However, the original MapReduce framework also needs some optimizations to improve its performance. In this paper, we propose a novel bloom filter based lightweight MapReduce index (BFMapReduce). Instead of scanning the whole dataset, our approach uses an auxiliary index to quickly skip unnecessary data segments, which can efficiently degrade the processing cost at map phase. Moreover, in order to deal with multi-dimension dataset, a converting schema is proposed. It can map multidimension data into one-dimension index. The experimental results show that our approach is efficient and lightweight. It can reduce the task running time dramatically with a little storage and maintenance cost.

show abstract

Bloom filter based optimization on HBase with MapReduce

Cited by 9 publications

References 8 publications

Improving Lookup and Query Execution Performance in Distributed Big Data Systems using Cuckoo Filter

Improving Lookup and Query Execution Performance in Distributed Big Data Systems using Cuckoo Filter

Spam analysis of big reviews dataset using Fuzzy Ranking Evaluation Algorithm and Hadoop

BF-MapReduce: A Bloom Filter Based Efficient Lightweight Search

Contact Info

Product

Resources

About