Fault-tolerant fault tolerance for component-based automation systems

Abstract

To guarantee high availability, automation systems must be fault-tolerant. To this end, they must provide redundant solutions for the critical parts of the system. Classical fault tolerance patterns such as standby or N-modular redundancy provide system stability in the case of a fault. Fault tolerance is subsequently degraded or, depending on the number of deployed replicas, often even unavailable until the system has been repaired.We introduce a combination of a component-based framework, redundancy patterns, and a runtime manager, which is able to provide fault tolerance, to detect host failures, and to trigger a reconfiguration of the system at runtime. This combined solution maintains system operation in case a fault occurs and automatically restores fault tolerance. The proposed solution is validated using a case study of an industrial distributed automation system. The validation shows how our solution quickly restores fault tolerance without the need for operator intervention or immediate hardware replacement while limiting the impact on other applications.

Right on Time Distributed Shared Memory

“…DCSs often execute control applications that are cyclic [11], [20], [21], i.e., run periodically. For example, an application might be required to periodically read values from a sensor and to activate "intensive cooling mechanisms" after t seconds of a machine exceeding a certain temperature threshold.…”

“…The most common monitoring scheme used by failure detectors in DCSs dictates that each host periodically, i.e., in every cycle, broadcasts a heartbeat message of some structure 2 [11], [12], [14], [25]- [27]. Based on these heartbeats and using time-outs, a failure detector monitors which hosts in the system have crashed and which have not.…”

“…These distributed data services are undeniably demanded in distributed control systems (DCSs) [6]- [11]. For example, power grid DCSs require real-time distributed storage systems (distributed hash tables) to store and retrieve monitoring data for wind and photo-voltaic generation sources [6]; ship-board DCSs require distributed real-time data services to be embedded within the ship [7], [8]; traffic control and agile manufacturing require the presence of distributed fresh data that reflects real-world status [7]- [10]; real-time execution platforms for DCSs, such as [11], rely on real-time replicated data structures for maintaining system and crash detection information.…”

“…To circumvent this impossibility, we propose a white box approach that utilizes existing services running within DCSs [11], [14], namely failure detection. DCSs usually employ failure detectors that provide means for detecting host crashes in real time [14]- [18].…”

“…We design TapeWorm, an algorithm that attaches itself to the crash monitoring messages (heartbeats) typically exchanged on a periodical basis to detect host crashes in DCSs [11], [12], [14]. TapeWorm uses the underlying heartbeats as the sole means of transporting information.…”