SUMMARYThe first Provenance Challenge was set up in order to provide a forum for the community to understand the capabilities of different provenance systems and the expressiveness of their provenance representations. To this end, a functional magnetic resonance imaging workflow was defined, which participants had to either simulate or run in order to produce some provenance representation, from which a set of identified queries had to be implemented and executed. Sixteen teams responded to the challenge, and submitted their inputs. In this paper, we present the challenge workflow and queries, and summarize the participants' contributions.

SUMMARYFor the first provenance challenge, we introduce a layered model to represent workflow provenance that allows navigation from an abstract model of the experiment to instance data collected during a specific experiment run. We outline modest extensions to a commercial workflow engine so it will automatically capture provenance at workflow runtime. We also present an approach to store this provenance data in a relational database. Finally, we demonstrate how core provenance queries in the challenge can be expressed in SQL and discuss the merits of our layered representation.

No abstract

Cloud Native Applications Overview An increased desire for rapid and scalable capability delivery has shifted focus to Cloud as an enabler of effective application development and deployment. The integration of Agile into a Cloud Native application life cycle process may not consistently allow an organization to more rapidly and reliably develop and scale applications. Product alignment concerns, and a lack of collaboration between operations and development teams, are often the primary culprits in today's increasingly complex Agency environments. So how can an Agency build, deliver, and operate resilient software systems more rapidly, at lower costs, and at scale to meet mission requirements? The answer may lie in an Agency's ability to adopt an integrated approach combining Agile methods, application architecture decisions, and modifications to an organization's core IT processes (depicted in F1.0), in addition to organizational and The Intersection of Agile Development and Cloud Platforms cultural shifts. Agencies should consider alignment of development staff to products, and infrastructure to offerings, optimizing resource allocation. An organization's capacity to execute these adjustments can significantly improve speed, scale, and resiliency objectives. This report presents both a strategy regarding how an organization can adopt an integrated approach into the Cloud Native application life cycle, and how it can effectively realize scale, speed, and resiliency objectives moving forward. F1.0 Critical Success Factors Agile Development Application Architecture Core IT Processes Organization and Culture Scale Resilience Speed Services-based API Management Event-driven dynamic self-healing microservices Cloud architectures Design patterns Separation of concerns Product teams Governance automation Continuous delivery Discovery and reuse About Deloitte Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee ("DTTL"), its network of member firms, and their related entities. DTTL and each of its member firms are legally separate and independent entities. DTTL (also referred to as "Deloitte Global") does not provide services to clients. Please see www.deloitte.com/about to learn more about our global network of member firms.

Abstract

No abstract

Abstract-Executing large number of independent jobs or jobs comprising of large number of tasks that perform minimal intertask communication is a common requirement in many domains. Various technologies ranging from classic job schedulers to latest cloud technologies such as MapReduce can be used to execute these "many-tasks" in parallel. In this paper, we present our experience in applying two cloud technologies Apache Hadoop and Microsoft DryadLINQ to two bioinformatics applications with the above characteristics. The applications are a pairwise Alu sequence alignment application and an EST (Expressed Sequence Tag) sequence assembly program. First we compare the performance of these cloud technologies using the above case and also compare them with traditional MPI implementation in one application. Next we analyze the effect of inhomogeneous data on the scheduling mechanisms of the cloud technologies. Finally we present a comparison of performance of the cloud technologies under virtual and non-virtual hardware platforms.

In our demonstration we present Trident, a scientific workflow workbench built on top of a commercial workflow system to leverage existing functionality to the extent possible. Trident is being developed in collaboration with the scientific computing community for use in a number of ongoing eScience projects that make use of scientific workflows, in particular the Pan-STARRS sky survey project and the Ocean Observatory Initiative. In our demonstration of Trident we will illustrate the ability to utilize both local and cloud resources for storage and execution, as well as services such as provenance, monitoring, logging and scheduling workflows over clusters. Our goal is to release Trident in early 2009 as an open source accelerator for others to use for eScience projects and to continue extending with support for new workflow features and services.