UNICORE 7 — Middleware services for distributed and federated computing

Benedyczak, Krzysztof; Schuller, Bernd; Sayed, Maria Petrova-El; Rybicki, Jedrzej; Grunzke, Richard

doi:10.1109/hpcsim.2016.7568392

Cited by 23 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the good readability of Python's syntax, the convenience and easy access for machine learning and data mining newcomers is increased (Chollet, 2018). Another example for a platform for shared computing and data resources is UNICORE 8 , which is a maturely developed software interface that provides access to a computing and data infrastructure including high-performance computing, clusters and file systems (Benedyczak et al, 2016).…”

Section: Discussionmentioning

confidence: 99%

A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics

et al. 2019

View full text Add to dashboard Cite

Machine learning tools represent key enablers for empowering material scientists and engineers to accelerate the development of novel materials, processes and techniques. One of the aims of using such approaches in the field of materials science is to achieve high-throughput identification and quantification of essential features along the process-structure-property-performance chain. In this contribution, machine learning and statistical learning approaches are reviewed in terms of their successful application to specific problems in the field of continuum materials mechanics. They are categorized with respect to their type of task designated to be either descriptive, predictive or prescriptive; thus to ultimately achieve identification, prediction or even optimization of essential characteristics. The respective choice of the most appropriate machine learning approach highly depends on the specific use-case, type of material, kind of data involved, spatial and temporal scales, formats, and desired knowledge gain as well as affordable computational costs. Different examples are reviewed involving case-by-case dependent application of different types of artificial neural networks and other data-driven approaches such as support vector machines, decision trees and random forests as well as Bayesian learning, and model order reduction procedures such as principal component analysis, among others. These techniques are applied to accelerate the identification of material parameters or salient features for materials characterization, to support rapid design and optimization of novel materials or manufacturing methods, to improve and correct complex measurement devices, or to better understand and predict fatigue behavior, among other examples. Besides experimentally obtained datasets, numerous studies draw required information from simulation-based data mining. Altogether, it is shown that experiment-and simulation-based data mining in combination with machine leaning tools provide exceptional opportunities to enable highly reliant Bock et al. Machine Learning in Materials Mechanics identification of fundamental interrelations within materials for characterization and optimization in a scale-bridging manner. Potentials of further utilizing applied machine learning in materials science and empowering significant acceleration of knowledge output are pointed out.

show abstract

Section: Discussionmentioning

confidence: 99%

A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics

et al. 2019

View full text Add to dashboard Cite

show abstract

“…They have used three layers for implementing security through a firewall, intrusion, and access control, which all are implemented in three layers. Kryszt Benedyczak et al have presented the use of middleware for implementing federated computing [36]. They have proposed a method that does not require either certificates or delegation mechanisms.…”

Section: Series Of Versions Of Thementioning

confidence: 99%

Improving the Open Stack Authentication system through federation with JASON Tokens

Basu¹

2019

IJATCSE

View full text Add to dashboard Cite

Many cloud computing systems are in place for providing different kinds of services, which include AWS, AZURE, Google Cloud, etc. which are proprietary. These clouds help the users implement their own IT requirements, but the users cannot configure or customize the cloud computing processes as per their needs, especially to handle the issue of security from the perspectives of authentication, access control, and data security.Open source cloud computing systems, which include Eucalyptus, Open Nebula, open stack, etc. allows the changes carried to the cloud computing systems primarily through configuration, the addition of API, the addition of processes, etc. Users can make changes to affect the system such that it works as per the user requirements, especially to improve the security system built into the cloud computing system, which sometimes found to be vulnerable for attack. Users are concerned about the security of their software data hosted on third-party IT infrastructure. Open Stack cloud computing platform is being used by many for implementing private clouds. Users can customize open stack as per their requirements.Open stack suffers from many security-related vulnerabilities that can be exploited by the users for attacking the user software and data. A review of the Open stack systems is required to find the gaps that are existing to plug the same. In this paper, a review of the Open stack presented, bringing out different kinds of vulnerabilities that exist in authenticating the users and a federation method using JSON tokens showed that help eliminating the Vulnerabilities existing in the open stack for enforcing security within the Keystone module of Open stack.

show abstract

“…Grid computing addresses this challenge with geographically distributed networked loosely coupled clusters. Popular grid computing middleware systems include Globus Toolkit and UNICORE (Uniform Interface to COmpute REsources). Ordinary PCs have been employed successfully for large scale scientific computing, most commonly using BOINC as middleware.…”

Section: Introductionmentioning

confidence: 99%

Resilient parallel computing on volunteer PC grids

Subhlok

Nguyen

Gabriel

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Volunteer PC hosts represent massive computation capacity at a low cost but are challenging to employ for general parallel computing. This paper presents the design, execution model, implementation, and evaluation of the Volpex framework for robust execution of parallel codes on volunteer PC grids characterized by system and network heterogeneity, varying availability, and frequent failures. The communication model is based on one-sided Put/Get calls to an abstract global shared space enhanced to support multiple autonomous instances of the same process at different stages of execution. Our approach customizes and combines the use of replication, checkpointing, and host selection. This presents formidable challenges that are addressed in this work; efficient checkpointing of distributed replicated processes, dynamic management of redundancy, quick restart in a distributed environment, and application specific host selection. The integrated runtime system is shown to effectively execute moderate size, coarse-grain, communicating codes on a worldwide distributed volunteer environment, a new milestone in volunteer computing. Extensive evaluation is conducted with example scientific codes on a pool of around 600 volunteer hosts. The results demonstrate the trade-offs in deploying checkpointing, redundancy, and host selection, and how these methods combine to provide application performance that is close to the ideal failure free performance. INTRODUCTIONDistributed computing is widely deployed for resource hungry applications that need massive computational power, large storage capacity, high processing speed or quick accessibility of data. Grid computing addresses this challenge with geographically distributed networked loosely coupled clusters. Popular grid computing middleware systems include Globus Toolkit 1 and UNICORE 2,3 (Uniform Interface to COmpute REsources). Ordinary PCs have been employed successfully for large scale scientific computing, most commonly using BOINC 4 as middleware. Web browsers can host middleware for distributed computing, a popular example being Weevilscout. 5The BOINC middleware uses volunteered public PCs for scientific applications when idle. It has been remarkably successful, managing as much as 5 Petaflops of aggregate compute power and supporting over 60 scientific research projects. However, this is a tiny fraction of the volunteer compute power that could be exploited. The research presented in this paper aims to enhance the state-of-the-art for parallel computing on volunteer PC grids. We will refer to PCs made available for scientific computing when idle as volunteer nodes (or hosts, or PCs) and an execution environment composed of PCs connected by a LAN or Internet as a volunteer PC grid (VPG); even if the PCs are managed hosts in an organization. Volunteer PCs represent a potentially immense but volatile resource; they are heterogeneous in terms of architecture, networking, and operating system, prone to failure, and their availability to execute guest scientific applications...

show abstract

UNICORE 7 — Middleware services for distributed and federated computing

Cited by 23 publications

References 19 publications

A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics

A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics

Improving the Open Stack Authentication system through federation with JASON Tokens

Resilient parallel computing on volunteer PC grids

Contact Info

Product

Resources

About