Dynamic Virtual Clustering with Xen and Moab

As larger and larger commodity clusters for high performance computing proliferate at research institutions around the world, challenges in maintaining effective use of these systems also continue to increase. Among the many challenges are maintaining the appropriate software stack for a broad array of applications, and sharing workload across clusters. The Dynamic Virtual Clustering (DVC) system integrates the Xen virtual machine with the Moab scheduler to allow for creation of virtual clusters on a per-job basis. These virtual clusters can provide a unique software environment for a particular application, or can provide a consistent software environment across multiple heterogeneous clusters. In this paper, the overhead of Xen-based DVC vs. native cluster performance is examined for workloads consisting of both serial and MPI-based parallel jobs.