Recovery for Failures in Rolling Upgrade on Clouds

Abstract

When cloud consumers perform rolling upgrade operations on cloud applications, they may encounter failures due to cloud uncertainty, interfering operations and incorrect configurations. For example, unreliable cloud API calls can make the rolling upgrade operation fail in unpredictable ways due to a long time delay to respond to the API call. This paper proposes two recovery strategies for recovering from rolling upgrade failures. The strategies are Compensated Undo & Redo and Reparation. We evaluated our recovery strategies on Asgard-based rolling upgrade operation on Amazon Cloud based on two evaluation metrics: MTTR and Service Performance. The experiment results show that our strategies perform better than the recovery mechanisms provided by Asgard itself. We also conduct a comparison between the two recovery strategies based on the metrics.

Towards a Taxonomy of Cloud Recovery Strategies

“…When analyzing and modeling sporadic cloud operations as processes, several existing process-oriented recovery methods [2][18] can be utilized for recovery. Generally, process-oriented recovery methods are similar to transaction-oriented recovery methods.…”

“…For a particular sporadic activity, such as upgrade, it can be conducted either manually or with the assistance of automation scripts such as Chef [6]. When cloud operators are conducting the sporadic activities either manually or automatically, recovery strategies such as process-oriented recovery methods [2] can be employed if runtime errors happen. Those recovery strategies are intended for sporadic activities during runtime phase.…”