Apache Airavata: Design and Directions of a Science Gateway Framework

Abstract

Abstract-This paper provides an overview of the Apache Airavata software system for science gateways. Gateways use Airavata to manage application and workflow executions on a range of backend resources (grids, computing clouds, and local clusters). Airavata's design goal is to provide component abstractions for major tasks required to provide gateway application management. Components are not directly accessed but are instead exposed through a client Application Programming Interface. This design allows gateway developers to take full advantage of Airavata's capabilities, and Airavata developers (including those interested in middleware research) to modify Airavata's implementations and behavior. This is particularly important as Airavata evolves to become a scalable, elastic "platform as a service" for science gateways. We illustrate the capabilities of Airavata through the discussion of usage vignettes. As an Apache Software Foundation project, Airavata's open community governance model is as important as its software base. We discuss how this works within Airavata and how it may be applicable to other distributed computing infrastructure and cyberinfrastructure efforts.

Cyberinfrastructure as a Platform to Facilitate Effective Collaboration between Institutions and Support Collaboratories

“…Gateways were originally considered to be browser-based Web portals for scientists to access remote computing resources through Grid middleware, but the software systems used to build gateways have matured to be considered first class cyberinfrastructure middleware in their own right. Projects such as Apache Airavata [24], HUBzero [25], CyVerse Agave [26], Galaxy [27], and others provide software and platform services for building gateways. Science gateways have also matured into production services for XSEDE, with the number of science gateway-based users surpassing traditional command-line users every quarterly reporting period since December 2013.…”

Integrating Apache Airavata with Docker, Marathon, and Mesos

“…Mesosphere provides a web interface to monitor the deployments and scale up services if needed. As Apache Airavata itself consists of multiple internal [2] and supporting components (the PGA, RabbitMQ, MySQL, etc. ), there is an opportunity to simplify deployment by dockerizing these components.…”

“…Apache Airavata [1,2] is a software framework that manages the execution and workflow of scientific computational long running jobs on distributed supercomputing environments for science gateways [3][4][5][6][7][8]. Airavata provides an Apache Thrift-based API on top of distributed computing middleware that assists with the management of jobs and workflows on a diverse set of remote computing and storage resources.…”

“…The Airavata server follows micro-service design model with functional components divided into Orchestrator, the Credential Store, the Airavata Registry, and an API server. These components are described in greater detail in [2]. The API Server is the entry point for gateways connecting to Airavata middleware and is based on Apache Thrift, which gives Airavata a strongly typed, programming language-independent way of defining its interfaces.…”

GSoC 2015 student contributions to GenApp and Airavata

“…An experiment can be a single application execution or a composite application with a sequence defined as a workflow. These components and their interactions are more fully described in [6]. Airavata is also serving as one of the bases of a hosted science gateway platform as a service, currently under development.…”