Testing embedded software: A survey of the literature

Abstract

Context: Embedded systems have overwhelming penetration around the world. Innovations are increasingly triggered by software embedded in automotive, transportation, medical-equipment, communication, energy, and many other types of systems. To test embedded software in an effective and efficient manner, a large number of test techniques, approaches, tools and frameworks have been proposed by both practitioners and researchers in the last several decades. Objective: However, reviewing and getting an overview of the entire state-of-the-art and the-practice in this area is challenging for a practitioner or a (new) researcher. Also unfortunately, as a result, we often see that many companies reinvent the wheel (by designing a test approach new to them, but existing in the domain) due to not having an adequate overview of what already exists in this area. Method: To address the above need, we conducted and report in this paper a systematic literature review (SLR) in the form of a systematic literature mapping (SLM) in this area. After compiling an initial pool of 588 papers, a systematic voting about inclusion/exclusion of the papers was conducted among the authors, and our final pool included 312 technical papers. Results: Among the various aspects that we aim at covering, our review covers the types of testing topics studied, types of testing activity, types of test artifacts generated (e.g., test inputs or test code), and the types of industries in which studies have focused on, e.g., automotive and home appliances. Furthermore, we assess the benefits of this review by asking several active test engineers in the Turkish embedded software industry to review its findings and provide feedbacks as to how this review has benefitted them. Conclusion: The results of this review paper have already benefitted several of our industry partners in choosing the right test techniques / approaches for their embedded software testing challenges. We believe that it will also be useful for the large worldwide community of software engineers and testers in the embedded software industry, by serving as an "index" to the vast body of knowledge in this important area. Our results will also benefit researchers in observing the latest trends in this area and for identifying the topics which need further investigations.

ConVRgence (VRIC) Virtual Reality International Conference Proceedings

“…It means that they first test the ADAS functionality with a complete numeric model including the environment (model-in-the-loop), then the embedded software (software-in-the-loop), the electronic control unit prototype (hardware-in-the-loop) and finally the behavior of humans in relation to the ADAS (human-in-the-loop or driver-in-the-loop). These different modeling phases describe the V-cycle development method (Garousi et al, 2018) including a battery of tests verifying requirements at each modelling level. The major drawback of this method is that the human is included last.…”

Software-testing education: A systematic literature mapping

“…To put things in perspective, we compare the annual trend of papers with the trend data for five other software testing areas as reported by five other SLM/SLR studies: (1) an SLR on testing embedded software [79], (2) an SLR on web application testing [80], (3) an SLR on Graphical User Interface (GUI) testing [81], (4) a survey on mutation testing [82], and (5) an SLR on software testability [1]. Note that, as we can see in Figure 5, the trend data for the other different SLRs end in different years, due to timeline differences in the execution and publication of those survey papers, e.g., the survey on mutation testing [82] was published in 2011 and thus only has the data until 2009.…”

Experimental Implementation of Power-Split Control Strategies in a Versatile Hardware-in-the-Loop Laboratory Test Bench for Hybrid Electric Vehicles Equipped with Electrical Variable Transmission

“…In this regard, convertibility of the control algorithms to C/C++ code assures that they are downloadable into digital signal processing (DSP) boards (e.g., dSPACE rapid prototyping product, "MiroLab Box") for processor-in-the-loop (PiL) tests as a next phase. The PiL phase considers hardware features and provides realistic situations by running the control algorithm for an emulated target behavior [7]. Hence, these steps can not only reveal coding failures, but also provide helpful insights for the hardware-in-the-loop (HiL) implementation where the vehicle behavior can be reproduced.…”

“…Hence, once an EMS is designed and verified in a closed-loop manner through HiL, it can ideally be transmitted to a vehicle-in-the-loop (ViL) testbed without design modifications, expecting that the proposed strategy can represent identical features. Considering the discussed phases, Figure 1 recapitulates the chronological design steps known as the "V" development process [7].…”