CLOUDRB: A framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud

Somasundaram, Thamarai Selvi; Govindarajan, K.

doi:10.1016/j.future.2013.12.024

Cited by 111 publications

(52 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Developments in this area can be found in the literature [31][32][33][34], where it becomes evident that there are many challenges to be addressed. Within INDIGO we focused (see Section 3.2) in the efficient sharing of resources among users following fair-share approaches (limiting the amount of resources that can be consumed by a user group), proper quota partitioning across different computing frameworks (like HPC and Cloud resources) or new Cloud computing execution models (like preemptible instances) as these are aspects that affect both users and resource providers.…”

Section: Context and State Of The Artmentioning

confidence: 99%

INDIGO-DataCloud: a Platform to Facilitate Seamless Access to E-Infrastructures

et al. 2018

View full text Add to dashboard Cite

This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications.

show abstract

Section: Context and State Of The Artmentioning

confidence: 99%

INDIGO-DataCloud: a Platform to Facilitate Seamless Access to E-Infrastructures

et al. 2018

View full text Add to dashboard Cite

show abstract

“…), it does have little or no impact on scalable systems acting on vertical scale (i.e., where the actual application is split into multiple processes or threads that jointly contribute to the application's functionality). Recently there have been more and more references to the so-called HPC Cloud [27]. However these either focus on making small parallel machines (for instance, in the order of 8 cores in Amazon EC), or provide access to thousands of cores in an almost unconnected fashion, similar to Compute Grids, (like Plura Processing).…”

Section: Cloud Computing Systemsmentioning

confidence: 99%

ElCore: Dynamic elastic resource management and discovery for future large-scale manycore enabled distributed systems

Zarrin

Aguiar

Barraca

2016

Microprocessors and Microsystems

View full text Add to dashboard Cite

The next generation of many-core enabled large-scale computing systems relies on thousands of billions of heterogeneous processing cores connected to form a single computing unit. In such large-scale computing environments, resource management is one of the most challenging, and complex issues for efficient resource sharing and utilization, particularly as we move toward Future ManyCore Systems (FMCS). This work proposes a novel resource management scheme for future peta-scale many-core-enabled computing systems, based on hybrid adaptive resource discovery, called ElCore. The proposed architecture contains a set of modules which will dynamically be instantiated on the nodes in the distributed system on demand. Our approach provides flexibility to allocate the required set of resources for various types of processes/applications. It can also be considered as a generic solution (with respect to the general requirements of large scale computing environments) which brings a set of interesting features (such as auto-scaling, multitenancy, multi-dimensional mapping, etc,.) to facilitate its easy adaptation to any distributed technology (such as SOA, Grid and HPC many-core). The achieved evaluation results assured the significant scalability and the high-quality resource mapping of the proposed resource discovery and management over highly heterogeneous, hierarchical and dynamic computing environments with respect to several scalability and efficiency aspects while supporting flexible and complex queries with guaranteed discovery results accuracy. The simulation results prove that, using our approach, the mapping between processes and resources can be done with high level of accuracy which potentially leads to a significant enhancement in the overall system performance.

show abstract

“…CLOUDRB [13] is a framework for scheduling and managing HPC scientific applications on clouds. In order to complete jobs within a user-specified deadline, CLOUDRB incorporates both deadlinebased and particle swarm optimization resource allocation.…”

Section: Related Workmentioning

confidence: 99%

“…hiding the complexities of workflow and resource allocation, researchers can focus purely on data analysis, as demonstrated in the case study. Second, Uncinus provides an interface which is focused on software, rather than hardware, similar to commonly used packages such as Galaxy [13]. Third, the resource selection algorithm makes use of attributes to turn a complex procedure into a simple question, how many resources are needed to run the application?…”

Section: Genomics Case Studymentioning

confidence: 99%