Transparent process migration: Design alternatives and the sprite implementation

Abstract

The Sprite operating system allows executing processes to be moved between hosts at any time. We use this process migration mechanism to offload work onto idle machines, and also to evict migrated processes when idle workstations are reclaimed by their owners. Sprite's migration mechanism provides a high degree of transparency both for migrated processes and for users. Idle machines are identified, and eviction is invoked, automatically by daemon processes. On Sprite it takes up to a few hundred milliseconds on SPARCstation 1 workstations to perform a remote exec, whereas evictions typically occur in a few seconds. The pmake program uses remote invocation to invoke tasks concurrently. Compilations commonly obtain speed‐up factors in the range of three to six; they are limited primarily by contention for centralized resources such as file servers. CPU‐bound tasks such as simulations can make more effective use of idle hosts, obtaining as much as eight‐fold speed‐up over a period of hours. Process migration has been in regular service for over two years.

Proactive process-level live migration and back migration in HPC environments

“…Process-level migration has been studied extensively in the past [32], [37], [46], [26], [4], [11], [15]. In the context of HPC, many MPI implementations have been retrofitted with or design for FT, ranging from automatic methods (checkpoint-based or log-based) [44], [41], [5] to nonautomated approaches [3], [17].…”

Mobile Objects and Agents (MOA)

“…When searching for an agent, the location object is first looked up at a local name server. If not found, it is looked When locating agents, different location schemes are used, similarly to those used in distributed operating systems, such as Charlotte [3], V kernel [31], Sprite [12] and Mach [23]. The MOA system supports: a) updating the home after agent moves, b) registering at a predefined name server, c) searching based on predefined itinerary and d) forwarding based on the trails left after migration (see Figure 9).…”

“…Process migration in the Sprite operating system supported the notion of a home node [12]. In the V Kernel process migration [31], migrating processes are located by searching them.…”

“…This was achieved at the cost of complexity introduced in the operating system [12,23,24]. Voyager also supports communication with the migrated away agent, but it relies only on the forwarding strategy.…”

“…tem (OS) level. There were many implementations of process and object migration [3,12,19,31], but none has achieved wide acceptance. Due to inherent complexity, it was hard to introduce process migration without impacting the stability and robustness of the underlying OS.…”