A Comparative Study of Containers and Virtual Machines in Big Data Environment

Abstract

Container technique is gaining increasing attention in recent years and has become an alternative to traditional virtual machines. Some of the primary motivations for the enterprise to adopt the container technology include its convenience to encapsulate and deploy applications, lightweight operations, as well as efficiency and flexibility in resources sharing. However, there still lacks an in-depth and systematic comparison study on how big data applications, such as Spark jobs, perform between a container environment and a virtual machine environment. In this paper, by running various Spark applications with different configurations, we evaluate the two environments from many interesting aspects, such as how convenient the execution environment can be set up, what are makespans of different workloads running in each setup, how efficient the hardware resources, such as CPU and memory, are utilized, and how well each environment can scale. The results show that compared with virtual machines, containers provide a more easy-to-deploy and scalable environment for big data workloads. The research work in this paper can help practitioners and researchers to make more informed decisions on tuning their cloud environment and configuring the big data applications, so as to achieve better performance and higher resources utilization.

Integration of Artificial Intelligence with Cloud Services

“…Always having access to provided services requires replicating and distributing data to more than one cloud server and that redundancy is one more service that is achieved through the cloud, backup. Containerization of applications is another focal cloud service [1]. Instead of virtualizing complete operating systems, containers are made from the application layer.…”

Prism

“…Application isolation and resource abstraction are critical for shared cluster provisioning. Containers runtimes such as Linux Containers, Docker and OpenVZ have emerged as lightweight performant alternatives to virtual machines [1]. Due to the increased scale and inherent complexity of such containerized clusters in production, these systems are frequently exposed to emerging phenomena directly impacting system performance.…”

“…Containerized clusters: Containers provide virtualized environments encapsulating applications and their configurations [1]. Similar to virtual machines (VMs), containers allow several application environments to share a single host machine.…”

“…Implemented as a kernel feature, containers do not require hypervisor hardware emulation and instead achieve resource isolation via resource multiplexing of kernel resources [11]. Containers provide several advantages over hypervisor-based virtualization, including; smaller image size [12], rapid boot time, and greater resource efficiency [1] allowing for rapid scaling. Hence, providers such Google Cloud [2] and AliYun [3] are increasingly leveraging containers in their physical computing infrastructure to form containerized clusters for Cloud services.…”

Efficient Docker Container Scheduling using ABC Optimization Technique

“…On the basis of different deployment models, cloud is divided into four types; namely, public clouds, private clouds, community clouds, and hybrid clouds. Enterprise software and distributed computing experiences novel languages, technologies, and platforms however after some time, many of them drop by wayside in cloud computing, Docker has brought a revolutionary change due to its different features [2,4]. The resources are delivered as services in cloud figuring.…”