A Scalable Signalling Mechanism for VM Migration with SR-IOV over Infiniband

Guay, Wei Lin; Reinemo, Sven-Arne; Johnsen, Bjorn Dag; Skeie, Tor; Torudbakken, Ola

doi:10.1109/icpads.2012.60

Cited by 3 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…14 [38,39]. This layer-two address change will force the peers communicating with the migrated VM to ask for the new path characteristics to reconnect, a behavior that leads us to the first Research Question (RQ) of this thesis: RQ 1: In dynamic cloud environments VMs can live migrate to optimize resource usage and performance.…”

Section: Research Challenges Addressed In This Thesismentioning

confidence: 99%

“…Ranadive considered those aspects, and developed tools and methods for monitoring and latency-aware scheduling of the VMs. Nonetheless, the closest related work to ours, that focuses in scalability and reconfiguration of lossless networks while live migrations are part of a cloud data center, is that of Guay [38,120]. In [38], Guay et al present a signaling mechanism that takes into account that all three IB addresses change when a VM is migrating, and notify the peers of the migrated VM to update their cached path information.…”

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

“…Nonetheless, the closest related work to ours, that focuses in scalability and reconfiguration of lossless networks while live migrations are part of a cloud data center, is that of Guay [38,120]. In [38], Guay et al present a signaling mechanism that takes into account that all three IB addresses change when a VM is migrating, and notify the peers of the migrated VM to update their cached path information. In our work, and specifically in Paper I, we first show that alternatively if one has the possibility to migrate all three IB addresses, no signaling is needed.…”

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

“…With the kernel bypassing in mind, the only suitable options to provide high performance networking in VMs are the direct device assignment techniques. For the aforementioned reasons, we, and other works related to IB and virtualization [30,29,38] chose to use SR-IOV for our experiments.…”

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

“…Moreover, some additional downtime and a measurable impact on the underlying network fabric and the SM will be introduced. The additional downtime and added overhead to the SM is due to the change of all three addresses of the IB SR-IOV VF [38]. The LID will change because the VM will be moved to a different physical host that is using a different LID and in current generation HCAs the LID of VMs is shared with the host LID [39].…”

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

See 4 more Smart Citations

A Novel Query Caching Scheme for Dynamic InfiniBand Subnets

Tasoulas

Gran

Johnsen³

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

Self Cite

View full text Add to dashboard Cite

The advent of the Internet of Things, sensor and social networks, to mention just a few examples, all contribute towards the solid establishment of the Big Data era. High Performance Computing (HPC) becomes necessary for the efficient processing of the massive amounts of data our society generates, and cloud computing is a critical component to deliver this processing power to a broader audience that cannot afford to acquire and maintain such complex computing systems themselves. However, HPC specific technology and performance is not yet apt to be delivered efficiently over highly flexible and dynamic environments, as typically are the virtualized cloud infrastructures.In this thesis, we address challenges that arise in high performance dynamic cloud environments, that are equipped with HPC specific technology, in the context of networking and virtualization. We use InfiniBand, a high performance lossless interconnection network as the basis of our research, and first show that lossless networks pose prime challenges when the nature of the infrastructure is very dynamic, i.e. exhibits continuous changes. Then we propose a network I/O virtualization architecture, the InfiniBand vSwitch architecture, that can make lossless network technologies more favorable in the cloud. Moreover, we propose different network reconfiguration methods to enable performance-driven reconfigurations in very large network topologies that are commonly found in data centers. Performance-driven reconfigurations are frequently needed to adapt to unpredictable workload changes resulting from the shared and on-demand nature of a cloud platform, or when cloud providers employ live migration of virtual machines to optimize the resource usage of their infrastructure. Last but not least, we propose a new Quality-of-Service metric, called delay, to capture the directly observable service degradation in consolidated cloud environments. We suggest that the delay can be used as a direct service level agreement metric between cloud providers and cloud tenants. AcknowledgementsFirst, I would like to express my gratitude to my three supervisors; Ernst Gunnar Gran, Tor Skeie and Kyrre Begnum. Without their positiveness, supervision, suggestions and encouragement, my PhD journey would not have been so fruitful and enjoyable during both the good and hard times. Then I would like to thank Bjørn Dag Johnsen from Oracle Norway, our closest collaborate in the ERAC project, the project that mainly funded this thesis, for his enthusiastic attitude and participation in the important discussions that shaped the direction of my work and introduced industrial relevance. My appreciation goes as well to Feroz Zahid for being the most easygoing colleague whom I could imagine sharing office with, and a brilliant and ambitious associate, and Sven-Arne Reinemo for his supervision during my early days.A special thank you must be directed to Hårek Haugerud, Anis Yazidi, Hugo Lewi Hammer, Laurence Marie Anna Habib and the whole NETSYS group at the Oslo and Akershus...

show abstract

Section: Research Challenges Addressed In This Thesismentioning

confidence: 99%

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

Section: Infiniband Sr-iov and Live Migrations In The Cloudmentioning

confidence: 99%

See 3 more Smart Citations

A Novel Query Caching Scheme for Dynamic InfiniBand Subnets

Tasoulas

Gran

Johnsen³

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

Self Cite

View full text Add to dashboard Cite

show abstract

Efficient routing and reconfiguration in virtualized HPC environments with vSwitch‐enabled lossless networks

Tasoulas

Zahid

Gran

et al. 2018

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

To meet the demands of communication-intensive workloads in the cloud, virtual machines (VMs) should utilize low overhead network communication paradigms. In general, such paradigms enable VMs to directly communicate with the hardware by means of a passthrough technology like Single-Root I/O Virtualization (SR-IOV). However, when passthrough-based virtualization is coupled with lossless interconnection networks, live migrations introduce scalability challenges due to the substantial network reconfiguration overhead. With these challenges in mind, we proposed a virtual switch (vSwitch) SR-IOV architecture for InfiniBand in our previous work titled "Towards the InfiniBand SR-IOV vSwitch Architecture". In this paper, we first suggest solutions to rectify the space-domain scalability issues that are present in vSwitch-enabled subnets as a result of the VMs using dedicated layer-two addresses. Then, we discuss routing strategies for virtualized environments using vSwitches and present a routing algorithm for Fat-Trees. We also present a reconfiguration method that minimizes imposed reconfiguration overhead on Fat-Trees.We perform an extensive evaluation of our prototype algorithms, and as vSwitch-enabled hardware does not yet exist, we deduce from empirical observations by emulating vSwitches with existing hardware, as well as large-scale simulations. Our results show significant reduction in the reconfiguration times as route recalculations can be eliminated, and for certain scenarios, the number of reconfiguration subnet management packets sent to switches is reduced from several hundred thousand down to a single one without degrading the routing quality.

show abstract