The Evolving Landscape of Software Performance Engineering

Abstract

Satisfactory software performance is essential for the adoption and the success of a product. In organizations that follow traditional software development models (e.g., waterfall), Software Performance Engineering (SPE) involves time-consuming experimental modeling and performance testing outside the actual production environment. Such existing SPE methods, however, are not optimized for environments utilizing Continuous Integration (CI) and Continuous Delivery (CD) that result in high frequency and high volume of code changes. We present a summary of lessons learned and propose improvements to the SPE process in the context of CI/CD. Our findings are based on SPE work on two products conducted over 5 years at a major online services company. We find that (a) SPE has mainly become a post hoc activity based on data from the production environment, (b) successful application of SPE techniques require frequent re-evaluation of priorities, and (c) engineers working on SPE require a broader skill set than one traditionally possessed by engineers working on performance. CCS CONCEPTS• Software and its engineering → Software post-development issues; Software performance; Programming teams.

Architectural Support for Software Performance in Continuous Software Engineering: A Systematic Mapping Study

“…As an example, we considered performance requirement as the primary problem and quality of service as the secondary problem for paper [SP58], while we considered performance analysis approach as main contribution and performance modelling approach and performance prediction approach as the secondary contributions for the paper [SP45]. The same approach was applied also for RQ 3 .…”

“…Researchers and industry practitioners have identified the importance of integrating performance engineering practices in continuous development processes in a timely and efficient way [1]. However, current software performance engineering methods are not tailored for environments using CSE processes and practices are lagging [2,3].…”