A Metrics Suite for Evaluating Flexibility and Complexity in Service Oriented Architectures

Abstract

Service Oriented Architecture (SOA) is emerging to be the predominant architectural style of choice for many organizations due to the promised agility, flexibility and resilience benefits. However, there are currently few SOA metrics designed to evaluate complexity, effort estimates and health status of SOA solutions. This paper therefore proposes a SOA metrics framework which includes both service level and SOA-wide metrics to measure design and runtime qualities of a SOA solution. The SOA-wide metrics predict the overall complexity, agility and health status of SOA solutions, while service level metrics focus on the fundamental building blocks of SOA, i.e. services. The combined views deliver a compelling suite of SOA metrics that would benefit organizations as they consider adopting SOA. These metrics, which are based on observations of many SOA engagements, are illustrated through a case study that describes a recent ongoing project at IBM where SOA was utilized to build the solution assets.

The dark side of SOA testing: Towards testing contemporary SOAs based on criticality metrics

“…Existing research [14] has already proposed a number of analogous complexity metrics for SOAs, including metrics such as the number of services, the number of versions per service, and the number of human tasks. However, these metrics do not aim at SOA testing.…”

“…Two simple ordinal metrics capturing this aspect are (1) the service statefulness (SST), i.e., whether the service's operations can be used in any order or whether a specific protocol is assumed, and (2) the whether the service requires a mediator for connecting to the message bus (IMR). Typically, stateful services are vastly more error-prone than stateless ones (see also [14]). Similarly, mediation is known to be complex and error-prone, hence, services that require mediation need to be tested with more care.…”

“…We argue that this is required to steer the testing of contemporary SOAs, as well as to support the definition of useful testbeds (i.e., to build a useful staging environment, one first needs to identify the most critical components that need to be modelled). To this end, we propose to adopt complexity metrics [14] from the software engineering domain, in order to capture component dependencies and identify critical components that have to be tested with particular care. We refer to these metrics as criticality metrics.…”

FIT for SOA? Introducing the F.I.T.-Metric to Optimize the Availability of Service Oriented Architectures

“…Most of the reviewed work deals with aspects and metrics to support the first four phases based on design-time data: Metrics to measure busi-ness alignment of SOA implementations [1,30], procedures to model [6,16] and implement SOA [3], including the prediction of development effort and implementation complexity early in the design phase [37]. Metrics to measure granularity, complexity and reuse [15,35,36], performance [5,13] and QoS [28,31] of SOAbased services also rely on design-time data. Most work on operation and optimization has been done on how to handle service level agreements primarily based on design-time data: how to formally describe them [19,34,39,40], technically implement, test and enforce them [4,10,15,17,18,23,25,26,32,33,42,44,45] or how to monitor them [2,20,21].…”

“…Metrics to measure granularity, complexity and reuse [15,35,36], performance [5,13] and QoS [28,31] of SOAbased services also rely on design-time data. Most work on operation and optimization has been done on how to handle service level agreements primarily based on design-time data: how to formally describe them [19,34,39,40], technically implement, test and enforce them [4,10,15,17,18,23,25,26,32,33,42,44,45] or how to monitor them [2,20,21]. Contributions available on SLA design deal with isolated approaches: Sauvé et al [38] and Marques et al [27] are in favor of deriving the service level targets directly from the business impact of the given service (i.e.…”