An Efficient Scalable Runtime System for Macro Data Flow Processing Using S-Net

Gijsbers, Bert; Grelck, Clemens

doi:10.1007/s10766-013-0271-8

Cited by 4 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead, S-Net borrows the idea of streaming networks of asynchronous, stateless components, which segregates applications into their natural building blocks and exposes the data flow between them. We have developed a highly tuned run-time system customized to the specific needs of S-Net [23]. In addition we have developed Distributed S-Net for cluster architectures [26].…”

Section: High-level Macro-dataflow Coordinationmentioning

confidence: 99%

Parallelization of Hierarchical Matrix Algorithms for Electromagnetic Scattering Problems

Larsson¹,

Zafari²,

Righero³

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Numerical solution methods for electromagnetic scattering problems lead to large systems of equations with millions or even billions of unknown variables. The coefficient matrices are dense, leading to large computational costs and storage requirements if direct methods are used. A commonly used technique is to instead form a hierarchical representation for the parts of the matrix that corresponds to far-field interactions. The overall computational cost and storage requirements can then be reduced to O(N log N). This still corresponds to a large-scale simulation that requires parallel implementation. The hierarchical algorithms are rather complex, both regarding data dependencies and communication patterns, making parallelization non-trivial. In this chapter, we describe two classes of algorithms in some detail, we provide a survey of existing solutions, we show results for a proof-of-concept implementation, and we provide various perspectives on different aspects of the problem. The list of authors is organized into three subgroups, Larsson and Zafari (coordination and proof-of-concept implementation), Righero, Francavilla, Giordanengo, Vipiana, and Vecchi (definition of and expertise relating to the application), Kessler, Ancourt, and Grelck (perspectives and parallel expertise).

show abstract

Section: High-level Macro-dataflow Coordinationmentioning

confidence: 99%

Parallelization of Hierarchical Matrix Algorithms for Electromagnetic Scattering Problems

Larsson¹,

Zafari²,

Righero³

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Boxes for S-Net are implemented in C. All S-Net experiments make use of the Front runtime system [27], which combines very low overhead of S-Net network maintenance with efficient transportation of records throughout the network.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…Assume that a worker needs to select a record for processing from the choice as defined by tuples (0, 2, 1), (0, 2, 2) and (1, X, X). Front would select the tuple (1, X, X), which represents a critical dependency whose elimination potentially increases the available parallelism [27].…”

Section: Experimental Evaluationmentioning

confidence: 99%

The Cost and Benefits of Coordination Programming: Two Case Studies in Concurrent Collections and S-NET

Zaichenkov

Tveretina

Shafarenko

et al. 2016

Parallel Process. Lett.

Self Cite

View full text Add to dashboard Cite

This is an evaluation study of the expressiveness provided and the performance delivered by the coordination language S-NET in comparison to Intel’s Concurrent Collections (CnC). An S-NET application is a network of black-box compute components connected through anonymous data streams, with the standard input and output streams linking the application to the environment. Our case study is based on two applications: a face detection algorithm implemented as a pipeline of feature classifiers and a numerical algorithm from the linear algebra domain, namely Cholesky decomposition. The selected applications are representative and have been selected by Intel researchers as evaluation testbeds for CnC in the past. We implement various versions of both algorithms in S-NET and compare them with equivalent CnC implementations, both with and without tuning, previously published by the CnC community. Our experiments on a large-scale server system demonstrate that S-Net delivers very similar scalability and absolute performance on the studied examples as tuned CnC codes do, even without specific tuning. At the same time, S-Net does achieve a much more complete separation of concerns between compute and coordination layers than CnC even intends to.

show abstract

“…Boxes for S-NET were implemented in C and were compiled with GNU GCC 4.6.3. All measurements were done with the FRONT runtime system [16], [17]. This is a novel runtime system, which combines a very low overhead of S-NET box/network instantiation with efficient transportation of records throughout the network and box replication.…”

Section: Performance Measurementsmentioning

confidence: 99%

A Case Study in Coordination Programming: Performance Evaluation of S-Net vs Intel's Concurrent Collections

Zaichenkov

Gijsbers

Grelck

et al. 2014

2014 IEEE International Parallel &Amp; Distributed Processing Symposium Workshops

Self Cite

View full text Add to dashboard Cite

Abstract-We present a programming methodology and runtime performance case study comparing the declarative data flow coordination language S-NET with Intel's Concurrent Collections (CnC). As a coordination language S-NET achieves a near-complete separation of concerns between sequential software components implemented in a separate algorithmic language and their parallel orchestration in an asynchronous data flow streaming network.We investigate the merits of S-NET and CnC with the help of a relevant and non-trivial linear algebra problem: tiled Cholesky decomposition. We describe two alternative S-NET implementations of tiled Cholesky factorization and compare them with two CnC implementations, one with explicit performance tuning and one without, that have previously been used to illustrate Intel CnC. Our experiments on a 48-core machine demonstrate that S-NET manages to outperform CnC on this problem.

show abstract

An Efficient Scalable Runtime System for Macro Data Flow Processing Using S-Net

Cited by 4 publications

References 13 publications

Parallelization of Hierarchical Matrix Algorithms for Electromagnetic Scattering Problems

Parallelization of Hierarchical Matrix Algorithms for Electromagnetic Scattering Problems

The Cost and Benefits of Coordination Programming: Two Case Studies in Concurrent Collections and S-NET

A Case Study in Coordination Programming: Performance Evaluation of S-Net vs Intel's Concurrent Collections

Contact Info

Product

Resources

About