Towards Haskell in the cloud

Epstein, Jeff; Black, Andrew P.; Peyton-Jones, Simon

doi:10.1145/2096148.2034690

Cited by 56 publications

(49 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If the initiating node either is not in the list of s group nodes or is already a member of the s group the function fails and an error is returned. When an s group name is not provided the crypto:strong rand bytes (30) function is used to generate a random s group name. The particular function was chosen as a proof of concept, and may be replaced by an alternative one that also guarantees high probability of name uniqueness.…”

Section: S Group Functionsmentioning

confidence: 99%

Improving the network scalability of Erlang

Chechina

Ghaffari

et al. 2016

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

As the number of cores grows in commodity architectures so does the likelihood of failures. A distributed actor model potentially facilitates the development of reliable and scalable software on these architectures. Key components include lightweight processes which 'share nothing' and hence can fail independently. Erlang is not only increasingly widely used, but the underlying actor model has been a beacon for programming language design, influencing for example Scala, Clojure and Cloud Haskell.While the Erlang distributed actor model is inherently scalable, we demonstrate that it is limited by some pragmatic factors. We address two network scalability issues here: globally registered process names must be updated on every node (virtual machine) in the system, and any Erlang nodes that communicate maintain an active connection. That is, there is a fully connected O(n 2 ) network of n nodes.We present the design, implementation, and initial evaluation of a conservative extension of Erlang -Scalable Distributed (SD) Erlang. SD Erlang partitions the global namespace and connection network using s groups. An s group is a set of nodes with its own process namespace and with a fully connected network within the s group, but only individual connections outside it. As a node may belong to more than one s group it is possible to construct arbitrary connection topologies like trees or rings.We present an operational semantics for the s group functions, and outline the validation of conformance between the implementation and the semantics using the QuickCheck automatic testing tool. Our preliminary evaluation in comparison with distributed Erlang shows that SD Erlang dramatically improves network scalability even if the number of global operations is tiny (0.01%). Moreover, even in the absence of global operations the reduced connection maintenance overheads mean that SD Erlang scales better beyond 80 nodes (1920 cores).

show abstract

Section: S Group Functionsmentioning

confidence: 99%

Improving the network scalability of Erlang

Chechina

Ghaffari

et al. 2016

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…Our second contribution is to demonstrate that our rewriting rules can be turned into an method to automatically synthesize a canonical sequentialization from a symmetric non-deterministic program (ğ 5.2). We use this synthesis algorithm to implement a distributed systems verification tool called Brisk 1 Brisk first compiles Haskell programs that use the Cloud Haskell library [Epstein et al 2011] into our core language (ğ 5.1). Brisk verifies that its input has only symmetric races, and computes its canonical sequentialization thereby checking absence of deadlocks and assertion failures.…”

Section: Synthesizing Sequentializationsmentioning

confidence: 99%

“…In this section, we present the syntax and semantics of IceT, a core language for representing message passing programs as in Erlang and Cloud Haskell [Epstein et al 2011]. Figure 10 shows the syntax of IceT.…”

Section: Message Passing Programsmentioning

confidence: 99%

Verifying distributed programs via canonical sequentialization

Bakst

Gleissenthall

Kıcı

et al. 2017

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

We introduce canonical sequentialization, a new approach to verifying unbounded, asynchronous, messagepassing programs at compile-time. Our approach builds upon the following observation: due the combinatorial explosion in complexity, programmers do not reason about their systems by case-splitting over all the possible execution orders. Instead, correct programs tend to be well-structured so that the programmer can reason about a small number of representative executions, which we call the program's canonical sequentialization. We have implemented our approach in a tool called Brisk that synthesizes canonical sequentializations for programs written in Haskell, and evaluated it on a wide variety of distributed systems including benchmarks from the literature and implementations of MapReduce, two-phase commit, and a version of the Disco distributed file-system. Brisk verifies unbounded versions of the benchmarks in tens of milliseconds, yielding the first concurrency verification tool that is fast enough to be integrated into a design-implement-check cycle. is limited to finite systems: when the number of processes is unbounded, e.g., if processes are spawned based on input parameters, model checking cannot guarantee correctness as the state space, and hence, the number of executions, is infinite. Consequently, the distributed systems developer is bereft of development-and compile-time tools that guarantee the absence of concurrency errors.Our Approach In this paper, we propose canonical sequentialization, a new approach to verifying concurrency properties of unbounded, asynchronous, message-passing programs at compile-time. Our approach builds upon the following observation: due the combinatorial explosion in complexity, programmers do not reason about their systems by case-splitting over all the possible execution orders. Instead, correct programs tend to be well-structured so that the programmer can reason about a small number of representative executions which we call the program's canonical sequentialization.Example: Two-Phase Commit Consider the classic two-phase commit protocol [Lampson and Sturgis 1976], which consists of a leader process trying to commit a transaction to a number of database nodes. The protocol proceeds in two phases. In the first phase, the leader issues a tentative transaction. The leader then waits for answers from the nodes who may decide to either accept or abort the transaction. This initiates the second phase: if all nodes agreed, the leader sends them a commit message; otherwise, it sends an abort message. Finally, each node sends its acknowledgement of the final decision.Even though leader and database nodes execute in parallel, the concurrency is well-structured. First, due to the lack of a shared memory, most actions executed by different nodes are independent of each other, and thus commute. For example, a tentative proposal sent to one of the database nodes is independent of the messages other database nodes might send or receive. Thus, in the spirit of Lipton's movers [Lipto...

show abstract

“…The domain specific languages (DSL) called Cloud Haskell, presented in [12], provides Erlang inspired remote evaluation for community cloud computing, allowing new processes to be spawned at remote locations. It also provides mechanisms for handling remote spawn function closures (free variables in functions), as discussed in Sec.…”

Section: Community Cloud Computingmentioning

confidence: 99%

Code management automation for Erlang remote actors

Francalanza

Zerafa

2013

Proceedings of the 2013 Workshop on Programming Based on Actors, Agents, and Decentralized Control

View full text Add to dashboard Cite

Distributed Erlang provides mechanisms for spawning actors remotely through its remote spawn BIF. However, for remote spawn to function properly, the node hosting the spawned actor must share the same codebase as that of the node launching the actor. This assumption turns out to be too strong for various distributed settings. We propose a higher-level framework for the remote spawn of sideeffect free actors, abstracting from and automating codebase migration and management.

show abstract

Towards Haskell in the cloud

Cited by 56 publications

References 16 publications

Improving the network scalability of Erlang

Improving the network scalability of Erlang

Verifying distributed programs via canonical sequentialization

Code management automation for Erlang remote actors

Contact Info

Product

Resources

About