DayDream: Executing Dynamic Scientific Workflows on Serverless Platforms with Hot Starts

Roy, Rohan Basu; Patel, Tirthak; Tiwari, Devesh

doi:10.1109/sc41404.2022.00027

Cited by 12 publications

(4 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Por outro lado, as aplicações científicas, geralmente modeladas como monólitos [Roy et al 2022b], podem ser representadas por grafos direcionados acíclicos (DAGs). Sistemas de gerenciamento de fluxos de trabalho (WMSs), como Pegasus, Airflow e Argo, simplificam sua implementação ao converterem o DAG abstrato em listas de tarefas priorizadas, posteriormente escalonadas por gerenciadores de recursos como Kubernetes, HTCondor ou Slurm.…”

Section: Motivaçãounclassified

“…No entanto, parte dessa complexidade não é comunicada do WMS para o gerenciador de recursos, levando a decisões de escalonamento não-ótimas [Lehmann et al 2023]. Além disso, ao escalonar tais aplicações em clusters de HPC com alocação estática de recursos, é comum observar tanto subutilização quanto superutilização desses recursos [Roy et al 2022b, Roy et al 2022a].…”

Section: Motivaçãounclassified

See 1 more Smart Citation

Escalonamento com Consciência Energética para Fluxos de Trabalho Científicos sem Servidor: Uma Abordagem de Aprendizado de Máquina

Rosa,

Goldman

2024

Anais Da XV Escola Regional De Alto Desempenho De São Paulo (ERAD-SP 2024)

View full text Add to dashboard Cite

Este artigo propõe abordar os desafios de eficiência energética e escalonamento de fluxos de trabalho científicos em ambientes de computação sem servidor. Integrando técnicas de aprendizado de máquina e simulação, a pesquisa visa preencher lacunas entre eficiência energética e escalonamento sem servidor. A metodologia inclui coleta de dados históricos, previsão de consumo energético por meio de aprendizado de máquina e desenvolvimento de políticas de escalonamento com redes neurais profundas. O projeto também envolve adaptação de sistemas de gerenciamento de fluxo de trabalho e validação em ambientes reais, visando oferecer soluções viáveis para os desafios atuais em HPC.

show abstract

Section: Motivaçãounclassified

Escalonamento com Consciência Energética para Fluxos de Trabalho Científicos sem Servidor: Uma Abordagem de Aprendizado de Máquina

Rosa,

Goldman

2024

Anais Da XV Escola Regional De Alto Desempenho De São Paulo (ERAD-SP 2024)

View full text Add to dashboard Cite

show abstract

“…The scheduler of a serverless computing platform can operate proactively and approximately anticipate the arrival of the user requests. For instance, Roy et al 153 proposed an approach to predict the function calls via fitting the invocation trend to a statistical distribution. Such prediction can help the platform to pre‐warm function containers and load their required data in speculation of upcoming task requests.…”

Section: Approximate Computing In the Serverless Cloudsmentioning

confidence: 99%

Efficiency in the serverless cloud paradigm: A survey on the reusing and approximation aspects

Denninnart,

Chanikaphon,

Amini Salehi

2023

Softw Pract Exp

View full text Add to dashboard Cite

SummaryServerless computing along with Function‐as‐a‐Service (FaaS) is forming a new computing paradigm that is anticipated to found the next generation of cloud systems. The popularity of this paradigm is due to offering a highly transparent infrastructure that enables user applications to scale in the granularity of their functions. Since these often small and single‐purpose functions are managed on shared computing resources behind the scene, a great potential for computational reuse and approximate computing emerges that if unleashed, can remarkably improve the efficiency of serverless cloud systems—both from the user's QoS and system's (energy consumption and incurred cost) perspectives. Accordingly, the goal of this survey study is to, first, unfold the internal mechanics of serverless computing and, second, explore the scope for efficiency within this paradigm via studying function reuse and approximation approaches and discussing the pros and cons of each one. Next, we outline potential future research directions within this paradigm that can either unlock new use cases or make the paradigm more efficient.

show abstract

“…In recent years, a wide range of applications have been adapted to FaaS architecture, including machine learning [14,15], big data analytics [16,17], Internet of Things [18,19], scientific computing [20,21], etc. In order to achieve high elasticity, applications developed and deployed on FaaS platforms usually consist of multiple different cloud functions, thus communications in the form of function call are required.…”

Section: Introductionmentioning

confidence: 99%

µFuncCache: A User-Side Lightweight Cache System for Public FaaS Platforms

Luo

et al. 2023

Electronics

View full text Add to dashboard Cite

Building cloud-native applications based on public “Function as a Service” (FaaS) platforms has become an attractive way to improve business roll-out speed and elasticity, as well as reduce cloud usage costs. Applications based on FaaS are usually designed with multiple different cloud functions based on their functionality, and there will be call relationships between cloud functions. At the same time, each cloud function may depend on other services provided by cloud providers, such as object storage services, database services, and file storage services. When there is a call relationship between cloud functions, or between cloud functions and other services, a certain delay will occur, and the delay will increase with the length of the call chain, thereby affecting the quality of application services and user experience. Therefore, we introduce μFuncCache, a user-side lightweight caching mechanism to speed up data access for public FaaS services, fully utilizing the container delay destruction mechanism and over-booked memory commonly found in public FaaS platforms, to reduce function call latency without the need to perceive and modify the internal architecture of public clouds. Experiments in different application scenarios have shown that μFuncCache can effectively improve the performance of FaaS applications by consuming only a small amount of additional resources, while achieving a maximum reduction of 97% in latency.

show abstract

DayDream: Executing Dynamic Scientific Workflows on Serverless Platforms with Hot Starts

Cited by 12 publications

References 71 publications

Escalonamento com Consciência Energética para Fluxos de Trabalho Científicos sem Servidor: Uma Abordagem de Aprendizado de Máquina

Escalonamento com Consciência Energética para Fluxos de Trabalho Científicos sem Servidor: Uma Abordagem de Aprendizado de Máquina

Efficiency in the serverless cloud paradigm: A survey on the reusing and approximation aspects

µFuncCache: A User-Side Lightweight Cache System for Public FaaS Platforms

Contact Info

Product

Resources

About