Abstract-The recent growth of interest for in-memory databases poses the question on whether established prediction methods such as response surfaces and simulation are effective to describe the performance of these systems. In particular, the limited dependence of in-memory technologies on the disk makes methods such as simulation more appealing than in the past, since disks are difficult to simulate. To answer this question, we study an in-memory commercial solution, SAP HANA, deployed on a high-end server with 120 physical cores. First, we apply experimental design methods to generate response surfaces that describe database performance as a function of workload and hardware parameters. Next, we develop a class-switching queueing network model to predict in-memory database performance under similar scenarios. By comparing the applicability of the two approaches to modeling multi-tenancy, we find that both queueing and response surface models yield mean prediction errors in the range 5%-22% with respect to mean memory occupancy and response times, but the accuracy for the latter deteriorates in response surfaces as the number of experiments are reduced, whereas simulation is effective in all cases. This suggests that simulation can be very effective in performance prediction for in-memory database management.
In-memory database systems are among the technological drivers of big data processing. In this paper we apply analytical modeling to enable efficient sizing of in-memory databases. We present novel response time approximations under online analytical processing workloads to model thread-level forkjoin and per-class memory occupation. We combine these approximations with a non-linear optimization program to minimize memory swapping in in-memory database clusters. We compare our approach with state-of-the-art response time approximations and trace-driven simulation using real data from an SAP HANA in-memory system and show that our optimization model is significantly more accurate than existing approaches at similar computational costs.
Big data processing is driven by new types of in-memory database systems. In this paper we apply performance modeling to efficiently optimize workload placement for such systems. In particular, we propose novel response time approximations for in-memory databases based on fork-join queuing models and contention probabilities to model variable threading levels and per-class memory occupation under analytical workloads. We combine these approximations with a non-linear optimization methodology that seeks for optimal load dispatching probabilities in order to minimize memory swapping and resource utilization. We compare our approach with state-of-the-art response time approximations using real data from an SAP HANA inmemory system and show that our models markedly improve accuracy over existing approaches, at similar computational costs.
Abstract-Predicting memory occupancy during the execution of large-scale analytical workloads becomes critical for in-memory databases. In particular, probabilistic performance measures for such systems are of interest, but difficult to model with analytical methods due to the highly variable threading levels in corresponding workloads. Since literature with queueing theoretic background largely ignores the memory modeling part, we propose a new probabilistic model to capture the memory occupancy distribution in such systems. We further combine this model with our analytical formulation TP-AMVA for greater efficiency compared to simulation and evaluate against experiments using SAP HANA.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.