Impact of Commodity Networks on Storage Disaggregation with NVMe-oF

Kashyap, Arjun; Gugnani, Shashank; Lu, Xiaoyi

doi:10.1007/978-3-030-71058-3_3

Cited by 2 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some work exists to evaluate the performance of disaggregated storage over NVMe-oF [28,29,36,38]. Klimovic et al [45,46] devise a remote flash storage system on top of Ethernet and discuss storage disaggregation architecture in [44].…”

Section: Related Workmentioning

confidence: 99%

“…Klimovic et al [45,46] devise a remote flash storage system on top of Ethernet and discuss storage disaggregation architecture in [44]. Kashyap et al [38] conduct a performance characterization of different NVMe-oF networking protocols and Jia et al [36] study the performance implications of NVMe-oF on ARM SoCs and Xu et al [71] evaluate local and remote I/O performance for applications running in containers. Guz et al [28,29] compare kernel space NVMe-oF target with iSCSI protocol.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

NVMe-oAF

Kashyap

2022

Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

Applications running inside containers or virtual machines, traditionally use TCP/IP for communication in HPC clouds and data centers. The TCP/IP path usually becomes a major performance bottleneck for applications performing NVMe-over-Fabrics (NVMe-oF) based I/O operations in disaggregated storage settings. We propose an adaptive communication channel, called NVMe-over-Adaptive-Fabric (NVMe-oAF), that applications could leverage to eliminate the high-latency and low-bandwidth incurred by remote I/O requests over TCP/IP. NVMe-oAF accelerates I/O intensive applications using locality awareness along with optimized shared memory and TCP/IP paths. The adaptiveness of the fabric stems from the ability to adaptively select shared memory or TCP channel and further applying optimizations for the chosen channel. To evaluate NVMe-oAF, we co-design Intel's SPDK library with our designs and show up to 7.1x bandwidth improvement and up to 4.2x latency reduction for various workloads over commodity TCP/IPbased Ethernet networks (e.g., 10Gbps, 25Gbps, and 100Gbps). We achieve similar (or sometimes better) performance when compared to NVMe-over-RDMA by avoiding the cumbersome management of RDMA in HPC cloud environments. Finally, we also co-design NVMe-oAF with H5bench to showcase the benefit it brings to HDF5 applications. Our evaluation indicates up to a 7x bandwidth improvement when compared with the network file system (NFS).

show abstract

Section: Related Workmentioning

confidence: 99%