A Novel Query Caching Scheme for Dynamic InfiniBand Subnets

Abstract

The advent of the Internet of Things, sensor and social networks, to mention just a few examples, all contribute towards the solid establishment of the Big Data era. High Performance Computing (HPC) becomes necessary for the efficient processing of the massive amounts of data our society generates, and cloud computing is a critical component to deliver this processing power to a broader audience that cannot afford to acquire and maintain such complex computing systems themselves. However, HPC specific technology and performance is not yet apt to be delivered efficiently over highly flexible and dynamic environments, as typically are the virtualized cloud infrastructures.In this thesis, we address challenges that arise in high performance dynamic cloud environments, that are equipped with HPC specific technology, in the context of networking and virtualization. We use InfiniBand, a high performance lossless interconnection network as the basis of our research, and first show that lossless networks pose prime challenges when the nature of the infrastructure is very dynamic, i.e. exhibits continuous changes. Then we propose a network I/O virtualization architecture, the InfiniBand vSwitch architecture, that can make lossless network technologies more favorable in the cloud. Moreover, we propose different network reconfiguration methods to enable performance-driven reconfigurations in very large network topologies that are commonly found in data centers. Performance-driven reconfigurations are frequently needed to adapt to unpredictable workload changes resulting from the shared and on-demand nature of a cloud platform, or when cloud providers employ live migration of virtual machines to optimize the resource usage of their infrastructure. Last but not least, we propose a new Quality-of-Service metric, called delay, to capture the directly observable service degradation in consolidated cloud environments. We suggest that the delay can be used as a direct service level agreement metric between cloud providers and cloud tenants. AcknowledgementsFirst, I would like to express my gratitude to my three supervisors; Ernst Gunnar Gran, Tor Skeie and Kyrre Begnum. Without their positiveness, supervision, suggestions and encouragement, my PhD journey would not have been so fruitful and enjoyable during both the good and hard times. Then I would like to thank Bjørn Dag Johnsen from Oracle Norway, our closest collaborate in the ERAC project, the project that mainly funded this thesis, for his enthusiastic attitude and participation in the important discussions that shaped the direction of my work and introduced industrial relevance. My appreciation goes as well to Feroz Zahid for being the most easygoing colleague whom I could imagine sharing office with, and a brilliant and ambitious associate, and Sven-Arne Reinemo for his supervision during my early days.A special thank you must be directed to Hårek Haugerud, Anis Yazidi, Hugo Lewi Hammer, Laurence Marie Anna Habib and the whole NETSYS group at the Oslo and Akershus...

Efficient routing and reconfiguration in virtualized HPC environments with vSwitch‐enabled lossless networks

“…11 The Shared Port architecture comes with a shared LID and Queue Pair (QP) † space resulting in two main shortcomings: (1) VFs are not configurable by the SM, meaning that features such as knowing the port state of the VFs or configuring individual routes per VF are unavailable. As shown in our previous work, 8 this LID change can seriously challenge the SM and the scalability of the subnet in the performance domain. As each LID is associated with a specific hypervisor and shared among all VMs residing on the hypervisor, a migrating VM will have its LID changed to the LID of the destination hypervisor.…”

“…In our previous work, 8 we migrate VMs with IB VFs while keeping all three addresses. The vGUID of the SR-IOV VF is migrated together with the VM, but the LID changes.…”

“…Another aspect is that with the currently implemented SR-IOV Shared Port model, the addresses of the VM will change after the migration, causing additional overhead in the SM and a negative impact on the scalable management and performance of the network fabric. 8 The latter aspect takes the perspective of the interconnect fabric and is the primary focus of this paper.…”

“…The traditional approach for connecting IB devices to VMs is by utilizing SR‐IOV with direct assignment, but achieving live migration of VMs assigned with IB Host Channel Adapters (HCAs) using SR‐IOV has shown to be challenging. () The identified challenges can be grouped into two categories: challenges related to the extended migration downtime due to the direct device assignment and challenges related to the scalability and operation of the underlying network. In this work, we focus on the second group of challenges and address issues from the perspective of the network.…”

“…2,4,7,8 The identified challenges can be grouped into two categories: challenges related to the extended migration downtime due to the direct device assignment and challenges related to the scalability and operation of the underlying network. 2,4,7,8 The identified challenges can be grouped into two categories: challenges related to the extended migration downtime due to the direct device assignment and challenges related to the scalability and operation of the underlying network.…”