Performance Monitoring in SAP HANA's Continuous Integration Process

Rehmann, Kim-Thomas; Seo, Changyun; Hwang, Dongwon; Truong, Binh Than; Boehm, Alexander; Lee, Dong Hun

doi:10.1145/2897356.2897362

Cited by 16 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Milestones: feedback loops between Dev and Ops, automated performance monitoring, key performance metrics in CI, Test and Ops, tools and performance metrics sharing across teams [5] 2015 ACM Development of a high performance computing system System design (based on DevOps principles) [25] 2016 ACM Implementation of CI in a specific database -SAP HANA Methods: performance benchmarks for pre-commit tests, exceptions, and restricting the amount of manual work for CI to a minimum [9] 2016 Elsevier Interaction data acquisition system for mashups…”

Section: Methodsmentioning

confidence: 99%

“…They can help develop real-time supply chain analytics decision systems [13], enable faster data processing, more flexible data access, more up-to-date data, less aggregates and deeper drilldown capabilities [2], reduce latency times and enable processing and analysis of large amounts of data [20,34] with many concurrent users [20]. Use cases showing the benefits of in-memory databases for emerging DevOps practices such as continuous integration are also starting to emerge [25]. The move to in-memory databases affects the hardware choice, the software architecture and the software development paradigms [24].…”

Section: Deriving Design Principles For An Alternative Database Systemmentioning

confidence: 99%

See 1 more Smart Citation

DevOps Continuous Integration: Moving Germany’s Federal Employment Agency Test System into Embedded In-Memory Technology

Sultanow¹,

Chircu²,

Jung³

et al. 2019

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

This paper describes the development of a continuous integration database test architecture for a highly important and large software application in the public sector in Germany. We apply action design research and draw from two emerging areas of research-DevOps continuous integration practices and inmemory database development-to define the problem, design, build and implement the solution, analyze challenges encountered, and make adjustments. The result is the transformation of a large test environment originally based on Oracle databases into a flexible and fast embedded in-memory architecture. The main challenges involved overcoming the differences between the SQL specifications supported by the development and production systems and optimizing the test runtime performance. The paper contributes to theory and practice by presenting one of the first studies showing a real-world implementation of a successful database test architecture that enables continuous integration, and identifying technical design principles for database test architectures in general.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Deriving Design Principles For An Alternative Database Systemmentioning

confidence: 99%

DevOps Continuous Integration: Moving Germany’s Federal Employment Agency Test System into Embedded In-Memory Technology

Sultanow¹,

Chircu²,

Jung³

et al. 2019

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…Performance is also important for users if they need to achieve certain response times or throughput for their target workload. Therefore, performance testing is part of pre-and post-submit testing for SAP HANA [71]. Developers create performance tests to measure the performance of their code, but also to protect against regressions in future updates.…”

Section: Performance Testingmentioning

confidence: 99%

Testing Very Large Database Management Systems: The Case of SAP HANA

et al. 2022

Self Cite

View full text Add to dashboard Cite

Software Testing is an established activity in the software development process to ensure and improve the quality of a software. Consequently, there exists a wide range of literature, popular information, and even multiple ISO standards covering this topic. However, we found that testing very large database management systems (DBMS) requires special adaptations of the generally available guidance for software testing and requires to solve specific challenges that may not be relevant for other areas or smaller software projects. We therefore discuss the testing of SAP HANA, a very large software project with millions of lines of code, to share insights about our approach, best practices, and unsolved challenges that are open for further research.

show abstract

“…However, metrics are not without problems, as assigning a desired threshold to an NFR is not trivial. Fixed or static thresholds may be problematic for complex NFRs, requiring alternative solutions such as desired, minimum, dynamic thresholds [53], or even the use of artificial intelligence to adapt the thresholds.…”

Section: Non-functional Requirements In Continuous Software Engineeri...mentioning

confidence: 99%

Continuously Managing NFRs: Opportunities and Challenges in Practice

Werner,

Li,

Lowlind

et al. 2021

Preprint

View full text Add to dashboard Cite

Non-functional requirements (NFR), which include performance, availability, and maintainability, are vitally important to overall software quality. However, research has shown NFRs are, in practice, poorly defined and difficult to verify. Continuous software engineering practices, which extend agile practices, emphasize fast paced, automated, and rapid release of software that poses additional challenges to handling NFRs. In this multi-case study we empirically investigated how three organizations, for which NFRs are paramount to their business survival, manage NFRs in their continuous practices. We describe four practices these companies use to manage NFRs, such as offloading NFRs to cloud providers or the use of metrics and continuous monitoring, both of which enable almost real-time feedback on managing the NFRs. However, managing NFRs comes at a cost-as we also identified a number of challenges these organizations face while managing NFRs in their continuous software engineering practices. For example, the organizations in our study were able to realize an NFR by strategically and heavily investing in configuration management and infrastructure as code, in order to offload the responsibility of NFRs; however, this offloading implied potential loss of control. Our discussion and key research implications show the opportunities, trade-offs, and importance of the unique give-and-take relationship between continuous software engineering and NFRs. Research artifacts may be found at https://doi.org/10.5281/zenodo.3376342.

show abstract

Performance Monitoring in SAP HANA's Continuous Integration Process

Cited by 16 publications

References 13 publications

DevOps Continuous Integration: Moving Germany’s Federal Employment Agency Test System into Embedded In-Memory Technology

DevOps Continuous Integration: Moving Germany’s Federal Employment Agency Test System into Embedded In-Memory Technology

Testing Very Large Database Management Systems: The Case of SAP HANA

Continuously Managing NFRs: Opportunities and Challenges in Practice

Contact Info

Product

Resources

About