UNICORE 6 — Recent and Future Advancements

Streit, Achim; Bała, Piotr; Beck-Ratzka, Alexander; Benedyczak, Krzysztof; Bergmann, Sandra; Breu, Rebecca; Daivandy, Jason Milad; Demuth, Bastian; Eifer, Anastasia; Giesler, André; Hagemeier, Björn; Holl, Sonja; Huber, V.; Lamla, Nadine; Mallmann, Daniel; Memon, Shahbaz; Memon, M. S.; Rambadt, M.; Riedel, Morris; Romberg, Mathilde; Schuller, Bernd; Schlauch, Tobias; Schreiber, Andreas; Soddemann, Thomas; Ziegler, Wolfgang

doi:10.1007/s12243-010-0195-x

Cited by 67 publications

(40 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…UNICORE is an open source Grid middleware, which provides uniform access to distributed, heterogeneous resources in order to perform computations and manage data [11]. It is implemented in Java and consists of services for job and data handling, security, workflows, and orchestration on the server side and of a graphical client, a command line client, and a client API.…”

Section: Unicorementioning

confidence: 99%

“…UIMA (Unstructured Information Management Architecture) defines standardized interfaces and allows for multithreading. In order to make use of large distributed computing resources grid middleware like Globus 5 and UNICORE [11] have been developed. For example UNICORE (Uniform Interface to Computing Resources) offers sophisticated workflow features as well as built-in application support.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Information Extraction From Chemical Patents

Bergmann

Romberg

2012

csci

Self Cite

View full text Add to dashboard Cite

The development of new chemicals or pharmaceuticals is preceded by an indepth analysis of published patents in this field. This information retrieval is a costly and time inefficient step when done by a human reader, yet it is mandatory for potential success of an investment. The goal of the research project UIMA-HPC is to automate and hence speed-up the process of knowledge mining about patents. Multi-threaded analysis engines, developed according to UIMA (Unstructured Information Management Architecture) standards, process texts and images in thousands of documents in parallel. UNICORE (UNiform Interface to COmputing Resources) workflow control structures make it possible to dynamically allocate resources for every given task to gain best cpu-time/realtime ratios in an HPC environment.

show abstract

Section: Unicorementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Information Extraction From Chemical Patents

Bergmann

Romberg

2012

csci

Self Cite

View full text Add to dashboard Cite

show abstract

“…Service grid (SG) systems based on gLite [1], UNICORE [2], Globus [3] and ARC [4] middleware are intensively used to solve parametric applications where the same code is executed with a large number of different parameter systems. The maintenance of such a grid system is quite expensive since they typically use managed clusters as computing element resources.…”

Section: Introductionmentioning

confidence: 99%

Efficient extension of gLite VOs with BOINC based desktop grids

Visegrádi

Kovács

Kacsuk

2014

Future Generation Computer Systems

View full text Add to dashboard Cite

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.-The EDGI project aims to integrate Service Grid and Desktop Grid systems.-MetaJob support enables the forwarding of multiple jobs to Desktop Grids through Service Grid systems -The MetaJob provides seamless job forward since the service grid system does not recognise it as multiple job.-Seamless job forward is the key to the performance issues caused by the submission of high-number of jobs.-MetaJob can be used in any current service grids like gLite, ARC, UNICORE. Highlights (for review) Efficient extension of gLite VOs with BOINC based desktop gridsÁdám Visegrádi, József Kovács*, Peter Kacsuk MTA SZTAKI, Kende u. 13-17, 1111 Budapest, Hungary {a.visegradi, smith, kacsuk}@sztaki.hu AbstractThe paper describes the results of the EU FP7 EDGI project concerning how to extend gLite VOs with public and institutional BOINC desktop grids. Beyond simply showing the integration architecture components and services, the main emphasis is on how this integrated architecture can efficiently support parameter study applications, based on the so-called metajob concept created by the EDGI project. The paper explains in detail how to use the metajob concept by gLite users to exploit the BOINC desktop grids connected to the gLite VO, as well as how metajobs are managed internally by the 3G Bridge service. Performance measurements show that the Metajob concept indeed can significantly improve the performance of gLite VOs extended with desktop grids. Finally, the paper describes the practical ways of connecting BOINC desktop grids to gLite VOs and the accounting mechanism in these integrated grid systems.

show abstract

“…The data analysis of this dataset has the goal to cluster special data points and identify any noise elements. In this paper we describe the architectural design and implementation necessary to run data analysis jobs on HPC resources through standards-based UNICORE middleware [10] and uses data from the Global Federated File System [28] that we derive from the architecture of the Extreme Science and Engineering Discovery Environment (XSEDE) [24]. UNICORE is a HPC middleware and deployed on production on XSEDE supercomputing sites, whereas…”

mentioning

confidence: 99%

“…The second variant is a command line client called UNICORE Command Line (UCC), which provides an interface for advanced users who know the low level details of writing job request scripts to be executed jointly on the batch system. For a more detailed architecture explanation we refer to [10].…”

mentioning

confidence: 99%

Enabling Scalable Data Processing and Management through Standards-based Job Execution and the Global Federated File System

Riedel¹,

Memon²,

Koeritz³

et al. 2016

SCPE

Self Cite

View full text Add to dashboard Cite

Abstract. Emerging challenges for scientific communities are to efficiently process big data obtained by experimentation and computational simulations. Supercomputing architectures are available to support scalable and high performant processing environment, but many of the existing algorithm implementations are still unable to cope with its architectural complexity. One approach is to have innovative technologies that effectively use these resources and also deal with geographically dispersed large datasets. Those technologies should be accessible in a way that data scientists who are running data intensive computations do not have to deal with technical intricacies of the underling execution system. Our work primarily focuses on providing data scientists with transparent access to these resources in order to easily analyze data. Impact of our work is given by describing how we enabled access to multiple high performance computing resources through an open standards-based middleware that takes advantage of a unified data management provided by the the Global Federated File System. Our architectural design and its associated implementation is validated by a usecase that requires massivley parallel DBSCAN outlier detection on a 3D point clouds dataset.Key words: UNICORE, Genesis II, statistical data mining, data processing, distributed file system, security, standards, parallel processing AMS subject classifications. 68M14 1.Introduction. An ever increasing number of datasets from scientific experimentation such as earth observatories or computational simulations generate an enormous amount of information for discovering useful knowledge. In order to analyze data, the area of statistical data mining provides useful methods and tools to extract and explore useful patterns or prediction models. The field of statistical data mining comes with intuitive methods to learn from data, using a wide variety of algorithms for clustering, classification and regression. Several implementations are available, for example, Matlab, R, Octave [3], or scikit-learn. Mostly, these tools offer serial implementation of the algorithms, which is quite challenging (i.e. insufficient memory, extremely long running times, etc.) for processing the volume of data having terabytes or petabytes of magnitude. Considering that amount, the resources running the data processing tools require large number of processors, as well as much more primary and secondary storage. Therefore, parallel tools and platforms such as Hadoop [15] implementing the map reduce paradigm [18] and selected massively parallel algorithm developments based on the MPI and OpenMP environments are commonly used.We observe mainly tools for (High Performance Computing) HPC and High Throuput Computing (HTC) paradigms evolving concurrently, but each supporting their own set of requirements. Scientific communities, either from biology, physics and medicine adopt more conservative approaches in order to retain their focus on scientific findings and as such traditional HPC environme...

show abstract

UNICORE 6 — Recent and Future Advancements

Cited by 67 publications

References 16 publications

Information Extraction From Chemical Patents

Information Extraction From Chemical Patents

Efficient extension of gLite VOs with BOINC based desktop grids

Enabling Scalable Data Processing and Management through Standards-based Job Execution and the Global Federated File System

Contact Info

Product

Resources

About