INDIGO-Datacloud: foundations and architectural description of a Platform as a Service oriented to scientific computing

Abstract

In this paper we describe the architecture of a Platform as a Service (PaaS) oriented to computing and data analysis. In order to clarify the choices we made, we explain the features using practical examples, applied to several known usage patterns in the area of HEP computing. The proposed architecture is devised to provide researchers with a unified view of distributed computing infrastructures, focusing in facilitating seamless access. In this respect the Platform is able to profit from the most recent developments for computing and processing large amounts of data, and to exploit current storage and preservation technologies, with the appropriate mechanisms to ensure security and privacy.

Orchestrating Complex Application Architectures in Heterogeneous Clouds

“…by implementing advanced scheduling strategies based on fair share or preemptible instances [46], at the layer of PaaS, e.g. by creating SLA-based orchestration components that support deployments on multi-Clouds [76] and, finally, at the layer of SaaS, e.g. by developing high-level REST and graphical user interfaces to facilitate the usage of computing infrastructures for different scientific communities [71].…”

“…The INDIGO-DataCloud orchestrator is the entry point to the PaaS layer, receiving TOSCA documents as input (step 4), to find the best match for the resource provisioning. The decision-making process is based on a SLA (Service Level Agreement) analysis and the assessment of the potential target providers availability (step 5), leveraging the INDIGO-DataCloud PaaS service stack [76,52]. One important feature of the orchestration system is that it supports hybrid deployments across different public or on-premises Cloud providers, making use of the VPN technology to establish secure and seamless connections among the compute nodes located in the different infrastructures (see an example TOSCA template for a hybrid deployment in [34]).…”

Improved Cloud resource allocation: how INDIGO-DataCloud is overcoming the current limitations in Cloud schedulers

“…However, in spite of its advantages, the cloud still has several areas that need to be improved in order to expose to the researchers its fully functional power. In this context, INDIGO-DataCloud 1 (INtegrating Distributed data Infrastructures for Global ExplOitation) [1], a project funded under the Horizon 2020 framework program of the European Union, addresses the challenge of developing an advanced data and computing platform targeted at scientific communities, deployable on multiple hardware, and provisioned over hybrid e-Infrastructures.…”