HyPer: A hybrid OLTP&amp;amp;OLAP main memory database system based on virtual memory snapshots

Kemper, Alfons; Neumann, Thomas

doi:10.1109/icde.2011.5767867

Cited by 459 publications

(349 citation statements)

References 21 publications

Supporting

Mentioning

342

Contrasting

Unclassified

Order By: Relevance

“…Shared-nothing deployments [57], based on fully independent (physically partitioned) database instances that collectively process the workload, are an increasingly appealing design even within a single node [32,55,58]. This is due to the scalability limitations of shared-everything systems, which suffer from contention when concurrent threads attempt to access shared resources [27].…”

Section: Shared-nothing Database Deploymentsmentioning

confidence: 99%

“…In addition, shared-nothing databases typically make better use of the available hardware resources whenever the workload executes transactions touching data on a single database instance. Systems such as H-Store [58] and HyPer [32] apply the shared-nothing design to the extreme, deploying one single-threaded database instance per CPU core. This enables simplifications or removal of expensive database components such as locking and latching.…”

Section: Shared-nothing Database Deploymentsmentioning

confidence: 99%

“…Innovations in OLTP continue to deserve significant attention, advocated by the recent emergence of appliances [46], startups 1 , and research projects (e.g. [15,28,32,33,39,47,58,67]). OLTP applications are mission-critical for many enterprises with little margin for compromising either performance or scalability.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Characterization of the Impact of Hardware Islands on OLTP

Porobic

Pandis

Branco³

et al. 2015

The VLDB Journal

View full text Add to dashboard Cite

Modern hardware is abundantly parallel and increasingly heterogeneous. The numerous processing cores have non-uniform access latencies to the main memory and processor caches, which causes variability in the communication costs. Unfortunately, database systems mostly assume that all processing cores are the same and that microarchitecture differences are not significant enough to appear in critical database execution paths. As we demonstrate in this paper, however, non-uniform core topology does appear in the critical path and conventional database architectures achieve suboptimal and even worse, unpredictable performance. We perform a detailed performance analysis of OLTP deployments in servers with multiple cores per CPU (multicore) and multiple CPUs per server (multisocket). We compare different database deployment strategies where we vary the number and size of independent database instances running on a single server, from a single shared-everything instance to fine-grained shared-nothing configurations. We quantify the impact of non-uniform hardware on various deployments by (a) examining how efficiently each deployment uses the available hardware resources and (b) measuring the impact of distributed transactions and skewed requests on different workloads. We show that no strategy is optimal for all cases and that the best choice depends on the combination of hardware topology and workload characteristics. Finally, we argue that transaction processing systems must be aware of the hardware topology in order to achieve predictably high performance.

show abstract

Section: Shared-nothing Database Deploymentsmentioning

confidence: 99%

Section: Shared-nothing Database Deploymentsmentioning

confidence: 99%

See 1 more Smart Citation

Characterization of the Impact of Hardware Islands on OLTP

Porobic

Pandis

Branco³

et al. 2015

The VLDB Journal

View full text Add to dashboard Cite

show abstract

“…In Section III we asses the impact of this conflict and describe how we resolve it using JiT-compiled query execution in HyPer [21]. In Section IV we describe the cost model and illustrate its usage for layout optimization in Section V. In Section VI we present our evaluation and conclude in Section VII.…”

Section: Introductionmentioning

confidence: 99%

CPU and cache efficient management of memory-resident databases

Pirk

Funke²,

Grund

et al. 2013

2013 IEEE 29th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-Memory-Resident Database Management Systems (MRDBMS) have to be optimized for two resources: CPU cycles and memory bandwidth. To optimize for bandwidth in mixed OLTP/OLAP scenarios, the hybrid or Partially Decomposed Storage Model (PDSM) has been proposed. However, in current implementations, bandwidth savings achieved by partial decomposition come at increased CPU costs. To achieve the aspired bandwidth savings without sacrificing CPU efficiency, we combine partially decomposed storage with Just-in-Time (JiT) compilation of queries, thus eliminating CPU inefficient function calls. Since existing cost based optimization components are not designed for JiT-compiled query execution, we also develop a novel approach to cost modeling and subsequent storage layout optimization. Our evaluation shows that the JiT-based processor maintains the bandwidth savings of previously presented hybrid query processors but outperforms them by two orders of magnitude due to increased CPU efficiency.

show abstract

“…Pure data stream management systems (e.g., [22]) can cope with high-rate input streams and rather complex queries; however, ad-hoc data analysis and advanced analytics on historical data are impossible. From the more traditional DBMS side, read-optimized data organization such as column-oriented data management or hybrid OLTP/OLAP solutions [23] provide efficient ad-hoc data analysis but exhibit drawbacks with regard to write-intensive applications. In the data warehouse area, real-time data warehouse approaches try to cope with a continuous stream of write-only updates and read-only queries at the same time [24].…”

Section: Related Workmentioning

confidence: 99%

Real-Time Business Intelligence in the MIRABEL Smart Grid System

Fischer

Kaulakienė

Khalefa

et al. 2013

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

Abstract. The so-called smart grid is emerging in the energy domain as a solution to provide a stable, efficient and sustainable energy supply accommodating ever growing amounts of renewable energy like wind and solar in the energy production. Smart grid systems are highly distributed, manage large amounts of energy related data, and must be able to react rapidly (but intelligently) when conditions change, leading to substantial real-time business intelligence challenges. This paper discusses these challenges and presents data management solutions in the European smart grid project MIRABEL. These solutions include realtime time series forecasting, real-time aggregation of the flexibilities in energy supply and demand, managing subscriptions for forecasted and flexibility data, efficient storage of time series and flexibilities, and realtime analytical query processing spanning past and future (forecasted) data. Experimental studies show that the proposed solutions support important real-time business intelligence tasks in a smart grid system.

show abstract

HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots

Cited by 459 publications

References 21 publications

Characterization of the Impact of Hardware Islands on OLTP

Characterization of the Impact of Hardware Islands on OLTP

CPU and cache efficient management of memory-resident databases

Real-Time Business Intelligence in the MIRABEL Smart Grid System

Contact Info

Product

Resources

About