Improving the network scalability of Erlang

Abstract

As the number of cores grows in commodity architectures so does the likelihood of failures. A distributed actor model potentially facilitates the development of reliable and scalable software on these architectures. Key components include lightweight processes which 'share nothing' and hence can fail independently. Erlang is not only increasingly widely used, but the underlying actor model has been a beacon for programming language design, influencing for example Scala, Clojure and Cloud Haskell.While the Erlang distributed actor model is inherently scalable, we demonstrate that it is limited by some pragmatic factors. We address two network scalability issues here: globally registered process names must be updated on every node (virtual machine) in the system, and any Erlang nodes that communicate maintain an active connection. That is, there is a fully connected O(n 2 ) network of n nodes.We present the design, implementation, and initial evaluation of a conservative extension of Erlang -Scalable Distributed (SD) Erlang. SD Erlang partitions the global namespace and connection network using s groups. An s group is a set of nodes with its own process namespace and with a fully connected network within the s group, but only individual connections outside it. As a node may belong to more than one s group it is possible to construct arbitrary connection topologies like trees or rings.We present an operational semantics for the s group functions, and outline the validation of conformance between the implementation and the semantics using the QuickCheck automatic testing tool. Our preliminary evaluation in comparison with distributed Erlang shows that SD Erlang dramatically improves network scalability even if the number of global operations is tiny (0.01%). Moreover, even in the absence of global operations the reduced connection maintenance overheads mean that SD Erlang scales better beyond 80 nodes (1920 cores).

Transparent fault tolerance for scalable functional computation

“…It would be intriguing to thoroughly explore the determinism properties of transparent fault tolerance in languages like HdpH-RS. The implementation could be improved in various ways: by integrating HdpH-RS with the the topology aware HdpH implementation (Section 6.4); by extending the communication layer to allow nodes to join a computation; and by avoiding the scalability limitation imposed by instantiating a fully connected graph of nodes during execution as in SD Erlang (Chechina et al, 2014). The current implementation re-evaluates any tasks created by a lost task, and this may be very expensive e.g.…”

Scaling Reliably

“…The Kalkyl results are discussed by [21], and Figure 29 in Section i shows representative results for distributed Erlang computing orbits with 2M and 5M elements on Athos. In all cases performance degrades beyond some scale (40 nodes for the 5M orbit, and 140 nodes for the 2M orbit).…”

“…This section outlines the Scalable Distributed (SD) Erlang libraries [21] we have designed and implemented to address the distributed Erlang scalability issues identified in Section ii. SD Erlang introduces two concepts to improve scalability.…”

“…S_group Semantics For precise specification, and as a basis for validation, we provide a small-step operational semantics of the s_group operations [21]. Figure 14 defines the state of an SD Erlang system and associated abstract syntax variables.…”

“…At the language level the design, implementation and validation of the new libraries (Section V) have been reported piecemeal [21,60], and are included here for completeness.…”