Opportunistic job processing of input data divided into partitions and distributed amongst task level managers via a peer-to-peer mechanism supplied by a cluster cache

A global-level manager access a work order from a client and parameters associated with the work order. A service level agreement to meet the work order parameters is determined. The service level agreement includes a price. An indication is received from the client that the service level agreement is accepted. The one or more input files are partitioned into multiple shards, and the work order into multiple jobs. The jobs are distributed among a plurality of clusters to be processed using underutilized computing resources in the clusters. The job outputs are combined to form the work order output. The jobs are monitored to insure that the deadline for completion of the work order will be met.