Parallel computing patterns for Grid workflows

Abstract

Whereas a consensus has been reached on defining the set ofworkflow patterns for business process modeling languages, no such patterns existsfor workflows applied to scientific computing on the Grid By looking at different kinds ofparallelism, in this paper we identify a set ofworkflow patterns related to parallel and pipelined execution. The paper presents how these patterns can be represented in different Grid workflow languages and discusses their implications for the design of the underlying workflow management and execution infrastructure. A preliminary classification ofthese patterns is introduced by surveying how they are supported by several existing advanced scientific and Grid workflow languages.

A Specification Language for Performance and Economical Analysis of Short Term Data Intensive Energy Management Services

“…Reuse of components allows developers to use knowledge of their properties to predict the new system properties [12]. (3) Workflow and data flow patterns have been widely studied [11,18]. More advanced data flow abstractions such as MapReduce, or Bulk Synchronous parallel (BSP) constitute the essence of parallel programming frameworks such as Hadoop and Hama.…”

“…There are many ways in which a system can be built to provide the same functionality with different concurrent behaviours and different deployments over distributed infrastructures [11]. Our methodological approach to identify the elements to compound our hybrid specification can be summarised with the equation 'Specification of CDFA = Functional Entities + Communication / Synchronisation mechanisms + Data Dependencies + Resources'.…”

Revenue Models for Streaming Applications over Shared Clouds

“…A stream workflow is composed of a sequence of stages and datasets are transmitted through the stages following the pipeline streaming model of computation [7]. Within a streaming based workflow, it is often useful to identify a "data acceptance rate", which identifies the rate at which a workflow stage can receive and process data.…”

Advanced data flow support for scientific grid workflow applications

“…However, no flexible dataset-oriented data flow is supported in either of these systems. Parallel Computing Patterns [14] identified a data parallelism pattern and its variants static/dynamic/adaptive data parallelism for Grid workflows. These patterns have a close relationship to the classical multi-instance workflow pattern [16] and its variants.…”

“…The source attribute of the sink data port inPfCol is set to sampleWf/inWf (line 11). In the data flow link (2), the source data port inPfCol with type of agwl:collection has a constraint agwl:distribution=BLOCK(2) (line [13][14], which specifies that the data elements of the collection (i.e. the files) are mapped pairwise to the corresponding parallel loop iterations and consumed by the activity activityA through its input data port inAct (line 23).…”