Container orchestration on HPC systems through Kubernetes

Zhou, Naweiluo; Georgiou, Yiannis; Pospieszny, Marcin; Zhong, Li; Zhou, Huan; Niethammer, Christoph; Pejak, Branislav; Marko, Oskar; Hoppe, Dennis

doi:10.1186/s13677-021-00231-z

Cited by 31 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, it removes the complexity of having to set up VMs and application runtimes by packaging the applications and runtimes as containers. This last point has been confirmed by Zhou et al [12], who investigated using Kubernetes and containers for HPC and found that it significantly improved portability and eased the installation of dependencies.…”

Section: Related Workmentioning

confidence: 57%

Performance Evaluation of Open-Source Serverless Platforms for Kubernetes

2022

View full text Add to dashboard Cite

Serverless computing has grown massively in popularity over the last few years, and has provided developers with a way to deploy function-sized code units without having to take care of the actual servers or deal with logging, monitoring, and scaling of their code. High-performance computing (HPC) clusters can profit from improved serverless resource sharing capabilities compared to reservation-based systems such as Slurm. However, before running self-hosted serverless platforms in HPC becomes a viable option, serverless platforms must be able to deliver a decent level of performance. Other researchers have already pointed out that there is a distinct lack of studies in the area of comparative benchmarks on serverless platforms, especially for open-source self-hosted platforms. This study takes a step towards filling this gap by systematically benchmarking two promising self-hosted Kubernetes-based serverless platforms in comparison. While the resulting benchmarks signal potential, they demonstrate that many opportunities for performance improvements in serverless computing are being left on the table.

show abstract

Section: Related Workmentioning

confidence: 57%

Performance Evaluation of Open-Source Serverless Platforms for Kubernetes

2022

View full text Add to dashboard Cite

show abstract

“…As a result, it can meet the needs of cloud-based industries. Kubernetes is also suitable for data-intensive jobs in cloud clusters and shows better performance for HPC clusters [153]. Furthermore, Kubernetes promotes innovation and advantages in future IoT and edge hardware.…”

Section: Kubernetes Was Initially Designed and Developed Bymentioning

confidence: 99%

Container Technologies for ARM Architecture: A Comprehensive Survey of the State-of-the-Art

et al. 2022

View full text Add to dashboard Cite

Container technology is becoming increasingly popular as an alternative to traditional virtual machines because it provides a faster, lighter, and more portable runtime environment for the applications. A container bundles the application and its binary code, libraries, and configuration files together while sharing the host operating system image. Accordingly, containers efficiently share resources and operate small micro-services, software programs, and even more extensive applications with less overhead than virtual machines. There are many container technologies available with Docker being the most popular and many technologies support multiple architectures, including the ARM architecture. Due to its energy efficiency and high-performance, which are crucial parameters in containerization, ARM architecture is becoming prevalent in container technologies. In this paper, we explore various container technologies that support ARM architecture and investigate the pros and cons of each technology. Moreover, we provide a comparative analysis of both container orchestrators and container runtimes that are most prominent competitors of Docker. We also consider security of container technologies with particular focus on the image scanning tools that supports ARM architecture. Our survey reveals that ARM technology is gaining popularity in containerization and almost all recent technologies support ARM architecture.

show abstract

“…The most recent research in this area [16] proposes a hybrid architecture of Kubernetes with Terascale Open-source Resource and QUEue Manager (TORQUE) for running and orchestration virtual HPC clusters over Kubernetes using usually the well-known Portable Batch System (PBS) notation. The proposed approach benefits from adding the TORQUE operator into the Kubernetes and utilizing both technologies, although, this adds significant complexity to the deployment.…”

Section: Related Workmentioning

confidence: 99%

Serverless High-Performance Computing over Cloud

Petrosyan

Astsatryan

2022

Cybernetics and Information Technologies

View full text Add to dashboard Cite

HPC clouds may provide fast access to fully configurable and dynamically scalable virtualized HPC clusters to address the complex and challenging computation and storage-intensive requirements. The complex environmental, software, and hardware requirements and dependencies on such systems make it challenging to carry out our large-scale simulations, prediction systems, and other data and compute-intensive workloads over the cloud. The article aims to present an architecture that enables HPC workloads to be serverless over the cloud (Shoc), one of the most critical cloud capabilities for HPC workloads. On one hand, Shoc utilizes the abstraction power of container technologies like Singularity and Docker, combined with the scheduling and resource management capabilities of Kubernetes. On the other hand, Shoc allows running any CPU-intensive and data-intensive workloads in the cloud without needing to manage HPC infrastructure, complex software, and hardware environment deployments.

show abstract

Container orchestration on HPC systems through Kubernetes

Cited by 31 publications

References 34 publications

Performance Evaluation of Open-Source Serverless Platforms for Kubernetes

Performance Evaluation of Open-Source Serverless Platforms for Kubernetes

Container Technologies for ARM Architecture: A Comprehensive Survey of the State-of-the-Art

Serverless High-Performance Computing over Cloud

Contact Info

Product

Resources

About