Automatic and portable performance modeling for parallel I/O: a machine-learning approach

A performance model for a parallel I/O system is essential for detailed performance analyses, automatic performance optimization of I/O request handling, and potential performance bottleneck identification. Yet how to build a portable performance model for parallel I/O system is an open problem. In this paper, we present a machine-learning approach to automatic performance modeling for parallel I/O systems. Our approach is based on the use of a platform-independent performance metamodel, which is a radial basis function neural network. Given training data, the metamodel generates a performance model automatically and efficiently for a parallel I/O system on a given platform. Experiments suggest that our goal of having the generated model provide accurate performance predictions is attainable, for the parallel I/O library that served as our experimental testbed on an IBM SP. This suggests that it is possible to model parallel I/O system performance automatically and portably, and perhaps to model a broader class of storage systems as well.