A Pthreads Wrapper for Fortran 2003

Awile, Omar; Sbalzarini, Ivo F.

doi:10.1145/2558889

Cited by 1 publication

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Designing the architecture around the event‐based coordination pattern means that we can ensure that tricky aspects such as this will not result in deadlock. The Forthreads module is used to wrap the pthreads C interface library, which effectively gives us full pthread functionality in Fortran. We found that it was important to decorate variables shared between different threads with the volatile attribute, which denotes that these might be modified by means not obvious from the code and hence not to rely on values held in cache.…”

Section: Monc In Situ Data Analyticsmentioning

confidence: 99%

In situ data analytics for highly scalable cloud modelling on Cray machines

Brown

Weiland

Hill

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

SummaryMONC is a highly scalable modelling tool for the investigation of atmospheric flows, turbulence, and cloud microphysics. Typical simulations produce very large amounts of raw data, which must then be analysed for scientific investigation. For performance and scalability reasons, this analysis and subsequent writing to disk should be performed in situ on the data as it is generated; however, one does not wish to pause the computation whilst analysis is carried out. In this paper, we present the analytics approach of MONC, where cores of a node are shared between computation and data analytics. By asynchronously sending their data to an analytics core, the computational cores can run continuously without having to pause for data writing or analysis. We describe our IO server framework and analytics workflow, which is highly asynchronous, along with solutions to challenges that this approach raises and the performance implications of some common configuration choices. The result of this work is a highly scalable analytics approach, and we illustrate on up to 32 768 computational cores of a Cray XC30 that there is minimal performance impact on the runtime when enabling data analytics in MONC and also investigate the performance and suitability of our approach on the KNL. convection scheme, 5,6 and cloud microphysics. 7,8 The simulations that these models run generate a significant amount of raw data; it is not this raw data itself that the scientists are most interested in but instead higher level information that results from analysis on this data. Previous generations of models, such as the LEM, which exhibited very limited parallel scalability, were able to perform this data analysis either by writing raw data to a file and analysing offline or by doing it in-line with the computation without much impact on performance. However, as modern models, such as MONC, open up the possibility of routinely running very large simulations on many thousands of cores, for performance and scalability, it is not possible to write this raw data to file and do analysis offline or stop the computation whilst analysis is performed in-line. This situation is likely to become more severe as we move towards exa-scale and run these models on hundreds of thousands of cores.In this paper, we introduce the data analysis framework approach and implementation that we have developed for MONC where, instead of computation, some cores of a processor run our IO server and are used for data analysis. The computation cores "fire and forget" their raw data to a corresponding IO server, which will then perform the analysis and any required IO. In order to promote this "fire and forget" approach, where computational cores can be kept busy doing their work, the IO server is highly asynchronous and has to deal with different data arriving at different times, which raises specific challenges. After discussing the context of MONC and related work by the community in more detail in Section2, Section 3 then focuses on our IO server, the analytics wo...

show abstract

Section: Monc In Situ Data Analyticsmentioning

confidence: 99%