Evaluating Dynamic Software Update Safety Using Systematic Testing

Abstract

Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime.To avoid failures due to the updating process itself, many DSU systems employ timing restrictions.However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions; con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe; and manual selection, which permits updates at developer chosen program points.We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail.We found that all three timing restrictions prevented most failures, but only manual selection allowed none. Further, although CFS and AS allowed many more update points, manual selection still supported updates with minimal delay. Finally, we found that manual selection required the least developer effort.Overall, we conclude that manual selection is most effective.

Mutable checkpoint-restart

“…To automatically quiesce a program in a stable and reproducible configuration, MCR requires instrumenting perthread quiescent points that specify safe locations to block long-lived threads-avoiding synchronization and updatesafety issues [29]-and yield short call stacks with minimal stack-resident state-avoiding pervasive stack instrumentation to trace stack-allocated data structures, a nontrivial source of overhead in prior solutions [38]. To fulfill both requirements, MCR selects blocking calls (e.g., accept) found at the top of long-running thread loops as ideal quiescent point candidates, similar to analogous update point-based strategies adopted in prior work in the area [30,41,42].…”

“…Quiescence detection algorithms proposed in prior work operate at the level of individual functions [7,10,22,25] or generic events [12,13,23,41,42,45]. The former approach is known for its weak consistency guarantees [23,29] and typically relies on passive stack inspection [7,10,22,25] that cannot guarantee convergence in bounded time [38,39]. The latter approach relies on either update-friendly system design [12,23,45]-rarely an option for existing C programs-or explicit per-thread update points [30,38,41,42]-typically annotated at the top of longrunning loops.…”

Automating object transformations for dynamic software updating

“…While this delay makes sense, it is not sufficient to avoid trouble. Hayden et al [7] studied several years' worth of changes to three server programs and found that dynamic updates derived from actual releases sometimes fail even while adhering to this "activeness" restriction. Other work [8] suggests that simply asking programmers to specify a few program points (dubbed update points) at which updates are permitted makes the system easier to reason about.…”

Specifying and Verifying the Correctness of Dynamic Software Updates

“…This section formalizes a semantics for dynamic updates to single-threaded programs, then defines the merging transformation and proves it correct with respect to the semantics. Many server programs for which dynamic updating is useful are single-threaded [12,18,11]. However, an important next step for this work would be to adapt it to support updates to multi-threaded (and distributed) programs.…”

“…For instance, Hayden et al observed that OpenSSH's test suite only grew between versions-all of the old tests continued to hold as time went on [11]. This makes intuitive sense: many updates simply add new features, leaving the old features (and properties about them) unchanged, or refactor the program to improve non-functional aspects such as performance.…”