Harnessing the Computing Continuum for Programming Our World

Beckman, Pete; Dongarra, Jack; Ferrier, Nicola; Fox, Geoffrey; Moore, Terry; Reed, Dan; Beck, Micah

doi:10.1002/9781119551713.ch7

Cited by 41 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A larger datacenter, at the other end of the spectrum, can be seen as one site. All those sites are more and more aggregated into what the emerging Edgeto-Cloud continuum [5,6,20], the final objective being to be able to operate such a continuum in a unified manner. SpecK participates to this objective, focusing on stream processing applications.…”

Section: Targeted Platformmentioning

confidence: 99%

SpecK: Composition of Stream Processing Applications over Fog Environments

Battulga

Miorandi

Tedeschi

2021

Distributed Applications and Interoperable Systems

View full text Add to dashboard Cite

Stream Processing (SP), i.e., the processing of data in motion, as soon as it becomes available, is a hot topic in cloud computing. Various SP stacks exist today, with applications ranging from IoT analytics to processing of payment transactions. The backbone of said stacks are Stream Processing Engines (SPEs), software packages offering a high-level programming model and scalable execution of data stream processing pipelines. SPEs have been traditionally developed to work inside a single datacenter, and optimised for speed. With the advent of Fog computing, however, the processing of data streams needs to be carried out over multiple geographically distributed computing sites: Data gets typically pre-processed close to where they are generated, then aggregated at intermediate nodes, and finally globally and persistently stored in the Cloud. SPEs were not designed to address these new scenarios. In this paper, we argue that large scale Fog-based stream processing should rely on the coordinated composition of geographically dispersed SPE instances. We propose an architecture based on the composition of multiple SPE instances and their communication via distributed message brokers. We introduce SpecK, a tool to automate the deployment and adaptation of pipelines over a Fog computing platform. Given a description of the pipeline, SpecKcovers all the operations needed to deploy a stream processing computation over the different SPE instances targeted, using their own APIs and establishing the required communication channels to forward data among them. A prototypical implementation of SpecK is presented, and its performance is evaluated over Grid'5000, a large-scale, distributed experimental facility.

show abstract

Section: Targeted Platformmentioning

confidence: 99%

SpecK: Composition of Stream Processing Applications over Fog Environments

Battulga

Miorandi

Tedeschi

2021

Distributed Applications and Interoperable Systems

View full text Add to dashboard Cite

show abstract

“…Also, new challenges arise for scientific applications to harness the computing continuum, as indicated in the work by Beckman et al [13], where they identify multiple infrastructures on which computing takes place such as interconnected sensors from IoT/Edge devices to computer clusters and Cloud infrastructures. Workflow-like applications may benefit from the orchestration of resources along the computing continuum.…”

Section: Introductionmentioning

confidence: 99%

Serverless Workflows for Containerised Applications in the Cloud Continuum

et al. 2021

View full text Add to dashboard Cite

This paper introduces an open-source platform to support serverless computing for scientific data-processing workflow-based applications across the Cloud continuum (i.e. simultaneously involving both on-premises and public Cloud platforms to process data captured at the edge). This is achieved via dynamic resource provisioning for FaaS platforms compatible with scale-to-zero approaches that minimise resource usage and cost for dynamic workloads with different elasticity requirements. The platform combines the usage of dynamically deployed auto-scaled Kubernetes clusters on on-premises Clouds and automated Cloud bursting into AWS Lambda to achieve higher levels of elasticity. A use case in public health for smart cities is used to assess the platform, in charge of detecting people not wearing face masks from captured videos. Faces are blurred for enhanced anonymity in the on-premises Cloud and detection via Deep Learning models is performed in AWS Lambda for this data-driven containerised workflow. The results indicate that hybrid workflows across the Cloud continuum can efficiently perform local data processing for enhanced regulations compliance and perform Cloud bursting for increased levels of elasticity.

show abstract

“…Deep Learning applications, implemented as distributed analytics, are currently limited by Cloud-centric models that suffer crippling latency limitations when the amount and frequency of data increases. The computational ecosystem that supports these analytics has become highly heterogeneous and geographically distributed, bringing significant challenges associated with the complexity and sustainability of performing decision-making on sensor data [4], [5]. In particular, many Deep Learning applications require important decision-making to be delivered in a timely manner [6], requiring a novel design that enables trade-offs between the time and the quality of analysis.…”

Section: Introductionmentioning

confidence: 99%

Enabling microservices management for Deep Learning applications across the Edge-Cloud Continuum

Houmani

Balouek-Thomert

Caron

et al. 2021

2021 IEEE 33rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

View full text Add to dashboard Cite

Deep Learning has shifted the focus of traditional batch workflows to data-driven feature engineering on streaming data. In particular, the execution of Deep Learning workflows presents expectations of near-real-time results with user-defined acceptable accuracy. Meeting the objectives of such applications across heterogeneous resources located at the edge of the network, the core, and in-between requires managing trade-offs between the accuracy and the urgency of the results. However, current data analysis rarely manages the entire Deep Learning pipeline along the data path, making it complex for developers to implement strategies in realworld deployments. Driven by an object detection use case, this paper presents an architecture for time-critical Deep Learning workflows by providing a data-driven scheduling approach to distribute the pipeline across Edge to Cloud resources. Furthermore, it adopts a data management strategy that reduces the resolution of incoming data when potential trade-off optimizations are available. We illustrate the system's viability through a performance evaluation of the object detection use case on the Grid'5000 testbed. We demonstrate that in a multi-user scenario, with a standard frame rate of 25 frames per second, the system speed-up data analysis up to 54.4% compared to a Cloud-only-based scenario with an analysis accuracy higher than a fixed threshold.

show abstract

Harnessing the Computing Continuum for Programming Our World

Cited by 41 publications

References 18 publications

SpecK: Composition of Stream Processing Applications over Fog Environments

SpecK: Composition of Stream Processing Applications over Fog Environments

Serverless Workflows for Containerised Applications in the Cloud Continuum

Enabling microservices management for Deep Learning applications across the Edge-Cloud Continuum

Contact Info

Product

Resources

About