Correctness attraction: a study of stability of software behavior under runtime perturbation

Abstract

Can the execution of a software be perturbed without breaking the correctness of the output? In this paper, we devise a novel protocol to answer this rarely investigated question. In an experimental study, we observe that many perturbations do not break the correctness in ten subject programs. We call this phenomenon "correctness attraction". The uniqueness of this protocol is that it considers a systematic exploration of the perturbation space as well as perfect oracles to determine the correctness of the output. To this extent, our findings on the stability of software under execution perturbations have a level of validity that has never been reported before in the scarce related work. A qualitative manual analysis enables us to set up the first taxonomy ever of the reasons behind correctness attraction.

Neutrality and epistasis in program space

“…This experiment shows high variability in the proportion of edits that do not compile, that are / are not neutral, and that are repairs. This is consistent with previous findings on differential robustness by statement type [2,6], but the source of the variance in our data is not immediately apparent.…”

“…For example, in biology, different mutation rates have evolved in different types of organisms, and mutation rates can vary widely even within a single chromosome [17] or at different times, as in affinity maturation when mutation rates are increased by up to five orders of magnitude. Baudry's work [2,6] suggests that both targeting specific locations and varying mutation rates could be advantageous in genetic improvement.…”

Software robustness: a survey, a theory, and prospects

“…The above-mentioned authors of antifragile software and code plasticity terminology published a paper together on "correctness attraction", as defined below: Definition 2.3 (Correctness Attraction [9]). An execution perturbation is a runtime change of the value of one variable in a statement or an expression.…”

“…Danglot et al [9] related the concept of correctness attraction to antifragility and robustness using correctness ratio, which they define as the "percentage of correct executions over the whole perturbation space". In particular, if, in their experiments, a program point had correctness ratio of 100% it was deemed antifragile.…”

TripleAgent: Monitoring, Perturbation and Failure-Obliviousness for Automated Resilience Improvement in Java Applications

“…c) Fault model: In this paper, we consider two fault models: 1) injecting only one exception, when the perturbation point is reached for the first time and 2) always injecting exceptions when the perturbation point is reached. d) Perturbation search space: We define the "perturbation search space" as the Cartesian product of all possible perturbation points and all fault models with respect to a workload [8], [9]. The size of the search space is the number of workload executions required to have an exhaustive picture of the behavior under perturbation.…”