A Survey of Large Scale Data Management Approaches in Cloud Environments

Sakr, Sherif; Liu, A.; Batista, Daniel Macêdo; Alomari, Mohammad

doi:10.1109/surv.2011.032211.00087

Cited by 363 publications

(202 citation statements)

References 50 publications

Supporting

Mentioning

185

Contrasting

Unclassified

Order By: Relevance

“…Multiple studies have explored various data management strategies using existing storage options for clouds [1], [2], [3], [31], [32], [33]. These general studies focus on scientific applications that process large, unstructured sets of static input data (i.e., data that is available in the cloud storage when the processing starts and remains unchanged during the computation).…”

Section: Related Workmentioning

confidence: 99%

“…This innovation is proving fundamental to the construction of systems capable of rapid scaling and high reliability. At the same time, however, as has been shown by multiple evaluation efforts [1], [2], [3], the performance trade-offs inherent in current virtualization technology mean that data-intensive applications are often not the best fit for infrastructure clouds and consequently cannot leverage the advantages of cloud computing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Streaming Strategies for Event Processing Across Infrastructure Clouds

Tudoran

Keahey²,

Riteau³

et al. 2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

Abstract-Infrastructure clouds revolutionized the way in which we approach resource procurement by providing an easy way to lease compute and storage resources on short notice, for a short amount of time, and on a pay-as-you-go basis. This new opportunity, however, introduces new performance trade-offs. Making the right choices in leveraging different types of storage available in the cloud is particularly important for applications that depend on managing large amounts of data within and across clouds. An increasing number of such applications conform to a pattern in which data processing relies on streaming the data to a compute platform where a set of similar operations is repeatedly applied to independent chunks of data. This pattern is evident in virtual observatories such as the Ocean Observatory Initiative, in cases when new data is evaluated against existing features in geospatial computations or when experimental data is processed as a series of time events. In this paper, we propose two strategies for efficiently implementing such streaming in the cloud and evaluate them in the context of an ATLAS application processing experimental data. Our results show that choosing the right cloud configuration can improve overall application performance by as much as three times.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evaluating Streaming Strategies for Event Processing Across Infrastructure Clouds

Tudoran

Keahey²,

Riteau³

et al. 2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

show abstract

“…A range of open source solutions are also available, including Hadoop and OpenStack. A number of specific innovation also underpin this work, including important work on computational frameworks, such as MapReduce, in areas of eventual consistency in large-scale systems, and in novel data storage architectures [10].…”

Section: Middleware Challenge 2: Cloud Computingmentioning

confidence: 99%

Middleware for Internet distribution in the context of cloud computing and the Internet of Things

2016

View full text Add to dashboard Cite

Middleware is software that resides between applications, services, and their underlying distributed architecture and platforms. Middleware provides several types of capabilities to developers, including providing higher-level programming abstractions to support the development of applications and services; supporting end-to-end quality attributes, such as scalability, persistence, and security; and masking problems such as system failure and heterogeneity of languages, operating systems, and networks.Middleware has provided a key set of layers in distributed architectures since the early 90s, starting with the basic client/server model of distribution. Over the past several decades, there have been extensive innovations in middleware capabilities, with new paradigms and underlying systems technologies emerging. Significant developments during this time frame include the programming models associated with distributed objects, components, and, These innovations have provided developers with an abundance of tools to develop distributed systems and, until recently, middleware has largely delivered on its promise of enhancing portability, interoperability, and integrability. Over the last few years, however, there have been significant trends in distributed systems that motivate new advances in middleware to overcome limitations with conventional middleware technologies. These trends include the emergence of the Internet of Things (IoT) and cloud computing, which add new and unprecedented challenges for scale, dependability, and security to the underlying middleware technology. When taken together, the challenges are even greater when we move toward complex distributed systems that are very large (Internet scale), highly heterogeneous, and dynamic. There is also a corresponding change in the deployment of distributed systems, moving from individual systems to systems of systems.To address the challenges for middleware emanating from the advent of the Internet of Things and cloud computing, new research is required, particularly with respect to the types of middleware needed to support key domains, such as cyber-physical systems, smart cities, the smart grid, big data analytics, digital earth, and so on. This special issue addresses these challenges by (i) considering the state-

show abstract

“…Of separate interest is the survey on systems for large-scale data management in the cloud [91], and Cattell's survey on NoSQL data stores [25]. Furthermore, the tutorials on Big Data and cloud computing [10] and on the I/O characteristics of NoSQL databases [92] as well as the work of Abadi on limitations and opportunities for cloud data management [1] are also related.…”

Section: Introductionmentioning

confidence: 99%

A survey of large-scale analytical query processing in MapReduce

2013

View full text Add to dashboard Cite

Enterprises today acquire vast volumes of data from different sources and leverage this information by means of data analysis to support effective decision-making and provide new functionality and services. The key requirement of data analytics is scalability, simply due to the immense volume of data that need to be extracted, processed, and analyzed in a timely fashion. Arguably the most popular framework for contemporary large-scale data analytics is MapReduce, mainly due to its salient features that include scalability, fault-tolerance, ease of programming, and flexibility. However, despite its merits, MapReduce has evident performance limitations in miscellaneous analytical tasks, and this has given rise to a significant body of research that aim at improving its efficiency, while maintaining its desirable properties.This survey aims to review the state-of-the-art in improving the performance of parallel query processing using MapReduce. A set of the most significant weaknesses and limitations of MapReduce is discussed at a high level, along with solving techniques. A taxonomy is presented for categorizing existing research on MapReduce improvements according to the specific problem they target. Based on the proposed taxonomy, a clas-C. Doulkeridis

show abstract

A Survey of Large Scale Data Management Approaches in Cloud Environments

Cited by 363 publications

References 50 publications

Evaluating Streaming Strategies for Event Processing Across Infrastructure Clouds

Evaluating Streaming Strategies for Event Processing Across Infrastructure Clouds

Middleware for Internet distribution in the context of cloud computing and the Internet of Things

A survey of large-scale analytical query processing in MapReduce

Contact Info

Product

Resources

About