Using Docker in high performance computing applications

Chung, Minh Thanh; Quang-Hung, Nguyen; Nguyen, Manh-Thin; Thoai, Nam

doi:10.1109/cce.2016.7562612

Cited by 88 publications

(47 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Others also put docker into high performance computing conditions [24] and compare it with virtual machines. Chung's work [24] evaluates the performance by running HPL benchmark which is compiled with OpenMPI and OpenBLAS and run with different problem sizes. According to their experimental results, docker container manages to have better performance than VMs while using less RAM.…”

Section: Related Workmentioning

confidence: 99%

Performance Evaluation of Deep Learning Tools in Docker Containers

Shi

Chu

2017

2017 3rd International Conference on Big Data Computing and Communications (BIGCOM)

View full text Add to dashboard Cite

With the success of deep learning techniques in a broad range of application domains, many deep learning software frameworks have been developed and are being updated frequently to adapt to new hardware features and software libraries, which bring a big challenge for end users and system administrators. To address this problem, container techniques are widely used to simplify the deployment and management of deep learning software. However, it remains unknown whether container techniques bring any performance penalty to deep learning applications. The purpose of this work is to systematically evaluate the impact of docker container on the performance of deep learning applications. We first benchmark the performance of system components (IO, CPU and GPU) in a docker container and the host system and compare the results to see if there's any difference. According to our results, we find that computational intensive jobs, either running on CPU or GPU, have small overhead indicating docker containers can be applied to deep learning programs. Then we evaluate the performance of some popular deep learning tools deployed in a docker container and the host system. It turns out that the docker container will not cause noticeable drawbacks while running those deep learning tools. So encapsulating deep learning tool in a container is a feasible solution.

show abstract

Section: Related Workmentioning

confidence: 99%

Performance Evaluation of Deep Learning Tools in Docker Containers

Shi

Chu

2017

2017 3rd International Conference on Big Data Computing and Communications (BIGCOM)

View full text Add to dashboard Cite

show abstract

“…Conforme pode ser observado, todos os testes tiveram significância (p<0.05), com exceção das execuções entre o Docker e o NFS síncrono (leitura e escrita) para discos rígidos paralelos sem concorrência. [Felter et al 2015] PXZ, LINPACK, STREAM, nuttcp, netperf, FIO, Redis e SysBench X X X X X [Ruiz et al 2015] NPB e TAU X X [Adufu et al 2015] autodock3 X [Amaral et al 2015] CPU-intensive, Sysbench e netperf X X X X [Beserra et al 2015] HPL e NetPIPE X X X [Joy 2015] JMeter X [Morabito et al 2015] Y-cruncher, NBENCH, Geekbench, noploop, Linpack, Bonnie++, Sysbench, IOzone, STREAM e netperf X X X X [Xavier et al 2015] Swingbench e Sysbench X X X [Chung et al 2016] HPL e Graph500 X X [Jaikar et al 2016] HTCondor X X X [Ruan et al 2016] SPEC CPU 2006, STREAM, FIO, netperf e HiBench X X X X [Herbein et al 2016] LINPACK X X X [Barik et al 2016] AIO Stress, Ram-speed, IOzone, Tbench, iperf, RuBBoS, Apache-Bench, Blake2, 7-zip e OpenSSL benchmark X X X X [Beserra et al 2015] fs test X X [Eiras et al 2016] ApacheBench e Stress Tool X X [Kozhirbayev and Sinnott 2017] Y-cruncher, LINPACK, Geekbench, Bonnie++, Sysbench, STREAM, netperf e iperf X X X X [Li et al 2017] Iperf, HardInfo, Bonnie++ e STREAM X X X X [Mavridis and Karatza 2017] LINPACK, STREAM, IO-zone e netperf X X X X [Lingayat et al 2018] Ferramentas do Docker e Linux X X X [Zeng et al 2017] Ferramentas do Docker e Linux X [Mizusawa et al 2018] Ferramentas do Docker e Linux X A tabela foi organizada pelo tipo de carga de trabalho utilizada e quais os recursos que foram testados: (P) Processamento, (M) Memória, (R) Rede, (D) Disco, (CV) Compartilhamento de Volume, (O) Outros, como cópia de arquivo, criação de processos, número de instâncias, entre outros.…”

Section: Resultsunclassified

Avaliação do Docker Volume e do NFS no Compartilhamento de Sistemas de Arquivos em Contêineres

Schaefer

Bachiega

Souza³

et al. 2019

Anais Do XX Simpósio Em Sistemas Computacionais De Alto Desempenho (SSCAD 2019)

View full text Add to dashboard Cite

A atual demanda por poder computacional exige que novos recursos sejam utilizados para aumentar a eficiência energética, reduzir custos operacionais, reduzir espaço nos centros de dados e, consequentemente, diminuir a emissão de carbono na atmosfera. Neste contexto, contêineres representam uma tecnologia de virtualização que tentam resolver essas demandas. Dessa forma, avaliar seu desempenho é fundamental para sua adoção em larga escala. Este artigo investiga, através de um experimento, o desempenho em termos de tempo de resposta de volumes para contêineres Docker Volume e NFS, visto que tal análise não foi encontrada nos trabalhos relacionados. Os resultados mostram que há diferença significativa entre todos os testes com o Docker Volume e o NFS assíncrono e alguns diferenças significativas com o NFS síncrono.

show abstract

“…Teknologi containers pada lingkungan Linux (Linux Envirovment) memberikan sistem yang terisolasi (isolated environment) pada level OS yang dijalankan pada satu induk linux kernel (host). Pada penelitian Minh Thanh Chung disebutkan fungsional containers ketika bekerja pada cloud computing memiliki banyak keuntungan dalam mengurangi overhead dan karakteristiknya yang ringan dibandingkan dengan teknologi hypervisor pada Virtual Machines (VMs) yang memvirtualisasikan dalam level abstraksi hardware, sedangkan containers adalah emulasi pada level abstraksi Operating System (OS) (Chung, 2016). Salah satu platform populer yang menerapkan teknologi containers yaitu Docker.…”

Section: Pendahuluanunclassified

Analisis Performansi Layanan FTP danVideo Streaming berbasis Network Function Virtualization menggunakan Docker Containers

2018

View full text Add to dashboard Cite

ABSTRAKInfrastruktur jaringan seperti router, secara tradisional menggunakan hardware yang bersifat proprietary. Teknologi virtualisasi pada fungsi jaringan atau NFV (Network Function Virtualization) membuat layanan ini dapat diimplementasikan sebagai aplikasi perangkat lunak yang dapat dijalankan di lingkungan virtual atau Virtualized Network Functions (VNFs). Selain menggunakan hypervisor (hardware-level virtualization), teknologi virtualisasi memiliki alternatif pengimplementasian dengan menggunakan teknologi containers (Operating system -level virtualization), salah satunya menggunakan Docker. Penelitian ini mengimplementasikan layanan FTP dan video streaming pada jaringan NFV di Docker Containers. Tanpa backgound traffic, layanan menunjukkan performansi QoS yang memenuhi standarisasi ITU-T G.1010 dengan delay FTP 0,12 ms dan delay video streaming 6,21 ms serta nilai packet loss kedua layanan sebesar 0%. Penggunaan CPU pada Docker ketika layanan dijalankan dibawah 1 %. Kata kunci: Virtualisasi, Containers, Docker, Network Function Virtualization, QoSABSTRACTNetwork infrastructure such as routers, traditionally using proprietary hardware. Virtualization technology on network function or NFV (Network Function Virtualization) makes this service can be implemented as a software application that can run in virtual environment or Virtualized Network Functions (VNFs). In addition to using hypervisor (hardware-level virtualization), virtualization technology has an alternative implementation using containers technology (Operating system-level virtualization), one of them using Docker. This research implements FTP and video streaming services on NFV networks in Docker Containers. Without background traffic, the service demonstrates QoS performance that meets the ITU-T G.1010 standardization with 0.12 ms FTP delay and 6.21 ms video streaming delay and with packet loss value of both services at 0%. CPU usage on Docker when service runs below 1%.Keywords: Virtualization, Containers, Docker, Network Function Virtualization, QoS

show abstract

Using Docker in high performance computing applications

Cited by 88 publications

References 15 publications

Performance Evaluation of Deep Learning Tools in Docker Containers

Performance Evaluation of Deep Learning Tools in Docker Containers

Avaliação do Docker Volume e do NFS no Compartilhamento de Sistemas de Arquivos em Contêineres

Analisis Performansi Layanan FTP danVideo Streaming berbasis Network Function Virtualization menggunakan Docker Containers

Contact Info

Product

Resources

About