Lilith lights: A network traffic visualization tool for high performance clusters

High performance computing increasingly utilizes loosely coupled clusters of commodity computers. Performance monitoring data on such platforms is traditionally performed by invasive message passing taps, and examined only after the run is complete. We present an alternative, designed to capitalize on the scalable nature of the platform. Lilith Lights is a non-invasive network and processor monitoring tool which collects and processes data from the individual cluster machines. The data is gathered and displayed concurrently with the running applications. The display format can correspond to the application's virtual topology or to the physical connectivity of the message passing hardware.