Testing of controllers with a large number of inputs and outputs suffers from the curse of dimensionality in terms of the combinatorial explosion of possible input combinations. In particular for conformance testing, this results in unbearable large numbers of test cases. To mitigate this problem, the plant feature approach has been proposed, which guarantees full coverage of the nominal behavior of the controller. This nominal behavior corresponds to the reachable state space when combining in a closed-loop the controller with its fault-free plant. Plant features are identified as intuitive basic knowledge about a system based on its physical behavior. Considering such physical limitations allows to reduce the scope of testing to the actual relevant behavior.
During system testing of automotive electrical control units various reasons can lead to invalid test failures, e.g., non-responding components, faulty simulation models, faulty test case implementations, or hardware or software misconfigurations. To determine whether a test failure is invalid and what the underlying cause was, the test executions have to be analyzed manually, which is tedious and therefore costly. In this work, we report the magnitude of the problem of invalid test failures with four system testing projects from the automotive domain. We find that up to 91% of failed test executions are considered invalid. An oftentimes overlooked challenge are unreliable test infrastructures which deteriorate the validity of the test runs. In the studied projects already between 27% and 53% of failed test executions are linked to unreliable test infrastructures.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.