Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Most previous workflow scheduling research focused on scheduling a single workflow on parallel systems. Recent researches show that utilizing idle time slots between scheduled tasks is a promising direction for efficient multiple workflow scheduling. Stavrinides and Karatza proposed a list scheduling approach to efficient utilization of the idle time slots through bin packing techniques. In this paper, we elaborate on this direction and develop a new approach to further improve multiple workflow scheduling performance through two techniques. The first, in contrast with the list scheduling approach, is clustering the tasks within workflows into groups before allocation. This can reduce inter-task communication cost and thus improve workflow execution performance. The second technique tries to make a balance between tasks' start time and the fitness of idle time slots when allocating task groups. The proposed approach has been evaluated with a series of simulation experiments and compared to the previous method. The results show that our approach outperforms the previous method significantly, up to 51% performance improvement in terms of average makespan.