Data Aggregation System - a system for information retrieval on demand over relational and non-relational distributed data sources

Ball, Gareth; Kuznetsov, Valentin; Evans, D.; Metson, S.

doi:10.1088/1742-6596/331/4/042029

Cited by 9 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RDF triples are indexed and stored in Hbase and MapReduce is used for pattern processing. Ball et al [8] with Data Aggregation System (DAS) present a single interface to collect and query relational and non-relational data from different sources. DAS uses MongoDB to provide a caching layer for data services along with storing the server logs, analytics data and key mapping.…”

Section: Related Workmentioning

confidence: 99%

An Evaluation of Cassandra for Hadoop

Dede¹,

Sendir²,

Kuzlu³

et al. 2013

2013 IEEE Sixth International Conference on Cloud Computing

View full text Add to dashboard Cite

Abstract-In the last decade, the increased use and growth of social media, unconventional web technologies, and mobile applications, have all encouraged development of a new breed of database models. NoSQL data stores target the unstructured data, which by nature is dynamic and a key focus area for "Big Data" research. New generation data can prove costly and unpractical to administer with SQL databases due to lack of structure, high scalability, and elasticity needs. NoSQL data stores such as MongoDB and Cassandra provide a desirable platform for fast and efficient data queries. This leads to increased importance in areas such as cloud applications, e-commerce, social media, bio-informatics, and materials science. In an effort to combine the querying capabilities of conventional database systems and the processing power of the MapReduce model, this paper presents a thorough evaluation of the Cassandra NoSQL database when used in conjunction with the Hadoop MapReduce engine. We characterize the performance for a wide range of representative use cases, and then compare, contrast, and evaluate so that application developers can make informed decisions based upon data size, cluster size, replication factor, and partitioning strategy to meet their performance needs.

show abstract

Section: Related Workmentioning

confidence: 99%

An Evaluation of Cassandra for Hadoop

Dede¹,

Sendir²,

Kuzlu³

et al. 2013

2013 IEEE Sixth International Conference on Cloud Computing

View full text Add to dashboard Cite

show abstract

“…Sun et al [29] show processing fast growing RDF datasets, indexed and stored in an HBase cluster, with Hadoop. Ball et al [8] present Data Aggregation System (DAS) to collect and query relational and non-relational datasets through a single interface. DAS uses MongoDB for various caching and logging operations.…”

Section: Related Workmentioning

confidence: 99%

Processing Cassandra Datasets with Hadoop-Streaming Based Approaches

Dede

Sendir

Kuzlu

et al. 2016

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

The progressive transition in the nature of both scientific and industrial datasets has been the driving force behind the development and research interests in the NoSQL model. Loosely structured data poses a challenge to traditional data store systems, and when working with the NoSQL model, these systems are often considered impractical and costly. As the quantity and quality of unstructured data grows, so does the demand for a processing pipeline that is capable of seamlessly combining the NoSQL storage model and a "Big Data" processing platform such as MapReduce. Although MapReduce is the paradigm of choice for data-intensive computing, Java-based frameworks such as Hadoop require users to write MapReduce code in Java while Hadoop Streaming module allows users to define non-Java executables as map and reduce operations. When confronted with legacy C/C++ applications and other non-Java executables, there arises a further need to allow NoSQL data stores access to the features of Hadoop Streaming. We present approaches in solving the challenge of integrating NoSQL data stores with MapReduce under non-Java application scenarios, along with advantages and disadvantages of each approach. We compare Hadoop Streaming alongside our own streaming framework, MARISSA, to show performance implications of coupling NoSQL data stores like Cassandra with MapReduce frameworks that normally rely on file-system based data stores. Our experiments also include Hadoop-C*, which is a setup where a Hadoop cluster is co-located with a Cassandra cluster in order to process data using Hadoop with non-java executables.

show abstract

“…Sun et al [22] show processing fast growing RDF datasets, indexed and stored in an HBase cluster, with Hadoop. Ball et al [7] present Data Aggregation System (DAS) to collect and query relational and non-relational datasets through a single interface. DAS uses MongoDB for various caching and logging operations.…”

Section: Data Processing (Mr2)mentioning

confidence: 99%

A Processing Pipeline for Cassandra Datasets Based on Hadoop Streaming

Dede

Sendir

Kuzlu

et al. 2014

2014 IEEE International Congress on Big Data

View full text Add to dashboard Cite

Abstract- 1The progressive transition in the nature of both scientific and industrial datasets has been the driving force behind the development and research interests in the NoSQL data model. Loosely structured data poses a challenge to traditional data store systems, and when working with the NoSQL model, these systems are often considered impractical and expensive. As the quantity of unstructured data grows, so does the demand for a processing pipeline that is capable of seamlessly combining the NoSQL storage model and a "Big Data" processing platform such as MapReduce. Although, MapReduce is the paradigm of choice for data-intensive computing, Java-based frameworks such as Hadoop require users to write MapReduce code in Java. Hadoop Streaming, on the other hand, allows users to define non-Java executables as map and reduce operations. Similarly, for legacy C/C++ applications and other non-Java executables, there is a need to allow NoSQL data stores access to the features of Hadoop Streaming. In this paper, we present approaches in solving the challenge of integrating NoSQL data stores with MapReduce for non-Java application scenarios, along with advantages and disadvantages of each approach. We compare Hadoop Streaming alongside our own streaming framework, MARISSA, to show performance implications of coupling NoSQL data stores like Cassandra with MapReduce frameworks that normally rely on file-system based data stores.

show abstract

Data Aggregation System - a system for information retrieval on demand over relational and non-relational distributed data sources

Cited by 9 publications

References 2 publications

An Evaluation of Cassandra for Hadoop

An Evaluation of Cassandra for Hadoop

Processing Cassandra Datasets with Hadoop-Streaming Based Approaches

A Processing Pipeline for Cassandra Datasets Based on Hadoop Streaming

Contact Info

Product

Resources

About