Complex Distributed Systems: The Need for Fresh Perspectives

Abstract

Distributed systems are at a watershed due to their increasing complexity. The heart of the problem is the extreme level of heterogeneity exhibited by contemporary distributed systems coupled with the need to be dynamic and responsive to change. In effect, we have moved from distributed systems to systems of systems. Following on from this, middleware is also at a watershed. The traditional view of middleware is no longer valid (i.e. as a layer of abstraction, masking the complexity of the underlying distributed system and providing a high-level programming model). In practice, middleware is often bypassed with complex systems constructed in a rather ad hoc manner as a mash-up of a variety of technologies. The end result is that middleware is no longer sure of its form or purpose and this lack of a viable approach is a huge barrier to the emergence of areas such as smart cities and emergency response systems. This paper argues that there is a need to fundamentally rethink the middleware landscape related to complex distributed systems. The core contribution of the paper is a set of fresh perspectives, which lead us in turn to novel principles and patterns for middleware and subsequently to new styles of platform. These perspectives include a move to emergent middleware, seeking flexible meta-structures for distributed systems, and a step away from generic to domain-specific technologies. A number of case studies are also presented to demonstrate what this might mean for future distributed systems.

The evolution of distributed computing systems: from fundamental to new frontiers

“…Related to paradigm specialization discussed in Section 5.1, the distributed systems research area appears to be at a particularly interesting cross-roads; ensure that system paradigms are designed to be generalizable to handle a wide variety of operational conditions and scenarios (at the cost of performance and efficiency), or alternatively focus on creating more specialized and bespoke distributed system more suitable to a particular task at the expense of generalization and portability. While the wide-spread adoption of the x86 architecture, middleware, and virtualization have reinforced that historically the community has championed generalizable and portability, continued diversification of paradigms and technological limitations have begun discussion whether the axis is pivoting in the other direction [55]. This is further reinforced by increasing customization of microprocessors, OSs, and power management techniques for particular use case scenarios.…”

Widening the Circle of Engagement Around Environmental Issues using Cloud-based Tools

“…By now, there is a rich tradition of developing cloud applications and complex distributed systems [55], specifically those handling predictive modelling and similar workloads. However, not all available tools and approaches are suitable, and forcing the latest SE trend would not necessarily obtain positive results.…”

STEER: Redes IoT Dirigidas por Intenções e Baseadas em SDN

“…As infraestruturas de IoT estão ficando cada vez mais complexas e difíceis de gerenciar. Um dos principais motivos é a alta volatilidade de sistemas distribuídos contemporâneos [Blair 2018], causada, notadamente, pelas mudanc ¸as constantes nas infraestruturas de rede. Assim, para se manterem funcionais e atenderem as demandas das aplicac ¸ões, as redes IoT precisam se adaptar em tempo de execuc ¸ão, levando em conta tanto os objetivos das aplicac ¸ões ao interagirem com a rede, quanto às suas próprias restric ¸ões, à medida que mudanc ¸as ocorrem em seu ambiente operacional.…”