Tracing and reexecuting operating system calls for reproducible performance experiments

This paper shows how system call traces can be obtained with minimal interference to the system being characterized, and used as realistic, repeatable workloads for experiments to evaluate operating system and file system designs and configuration alternatives. Our system call trace mechanism, called ULTra, captures a complete trace of each UNIX process’s calls to the operating system. The performance impact is normally small, and it runs in user mode without special privileges. Traces can be rerun in two ways: the operating system activity can be reproduced by simply replaying the system calls interspersed with appropriate delays. More interestingly, we also show how the resulting traces can be used to drive full, repeatable reexecution of the captured behaviour. The paper concludes with an evaluation and comparison of the usefulness and accuracy of these techniques for predicting the performance impact of system configuration alternatives. We present two case studies, examining the effect of file system caching on a WWW server’s performance, and the performance benefit of using a local disk instead of an NFS fileserver.