An empirical evaluation of flow based programming in the machine learning deployment context

Paleyes, Andrei; Cabrera, Christian; Lawrence, Neil D.

doi:10.1145/3522664.3528601

Cited by 7 publications

(26 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This obstacle can negatively impact data-driven components (e.g., ML-based components) as they perform worse when the quality of the data they process is poor. Additional efforts are needed to create external mechanisms (e.g., middlewares) that monitor systems' data quality and adapt individual components accordingly [91]. Recognizing this drawback of SOA that impedes faster and smoother deployments of data-driven ML solutions, the community has proposed multiple strategies to bridge this gap, such as data meshes [37] or domain-oriented SOA [52].…”

Section: Data-oriented Architectures Paradigm and ML Challengesmentioning

confidence: 99%

“…Components in current software design paradigms (e.g., microservices or objects) usually fall into "The Data Dichotomy" as they hide their data, while data management requires exposing data. The dichotomy does not suit data-related tasks as additional efforts are needed to access the data that flows through the system [91].…”

Section: Data As a First Class Citizenmentioning

confidence: 99%

“…Systems use streams when data is generated and needs to be analysed in real-time. We observed that the use of streams is particularly common for systems that are built with dataflow architecture [32,91], where data inputs are transformed by different components while they flow through the system [14,23,31,36,56,59,60,72,75,83,99,102,112,116,125]. A good example of system architected in that way is described by Sultana et al [116].…”

Section: Data As a First Class Citizenmentioning

confidence: 99%

“…However, ML algorithms require data to learn, leading to a tight coupling between data and logic, thus violating some of the principles of modern software systems design. Consequently, data-related tasks require additional efforts from developers and data scientists when working with systems built with existing paradigms [91]. architectures, enabling technologies, and adaptation techniques.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Real-world Machine Learning Systems: A survey from a Data-Oriented Architecture Perspective

Cabrera¹,

Paleyes²,

Thodoroff³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the upsurge of interest in artificial intelligence machine learning (ML) algorithms, originally developed in academic environments, are now being deployed as parts of real-life systems that deal with large amounts of heterogeneous, dynamic, and high-dimensional data. Deployment of ML methods in real life is prone to challenges across the whole system life-cycle from data management to systems deployment, monitoring, and maintenance. Data-Oriented Architecture (DOA) is an emerging software engineering paradigm that has the potential to mitigate these challenges by proposing a set of principles to create data-driven, loosely coupled, decentralised, and open systems. However DOA as a concept is not widespread yet, and there is no common understanding of how it can be realised in practice. This review addresses that problem by contextualising the principles that underpin the DOA paradigm through the ML system challenges. We explore the extent to which current architectures of ML-based real-world systems have implemented the DOA principles. We also formulate open research challenges and directions for further development of the DOA paradigm.

show abstract

Section: Data-oriented Architectures Paradigm and ML Challengesmentioning

confidence: 99%

Section: Data As a First Class Citizenmentioning

confidence: 99%

Section: Data As a First Class Citizenmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Real-world Machine Learning Systems: A survey from a Data-Oriented Architecture Perspective

Cabrera¹,

Paleyes²,

Thodoroff³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently it has been observed that there is a strong connection between dataflow graphs and structured causal models [22]. Importantly, it was noted that the dataflow systems produce graphs that are complete by design.…”

Section: Introductionmentioning

confidence: 99%

Causal fault localisation in dataflow systems

Paleyes

Lawrence

2023

Proceedings of the 3rd Workshop on Machine Learning and Systems

View full text Add to dashboard Cite

Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines. CCS CONCEPTS• Software and its engineering → Error handling and recovery.

show abstract