Dynamic server assignment in an extended machine-repair model

This article considers an extension of the classic machine-repair problem. The machines, apart from receiving service from a single repairman, now also supply service themselves to queues of products. The extended model can be viewed as a two-layered queueing network, in which the queues of products in the first layer are generally correlated, due to the fact that the machines have to share the repairman's capacity in the second layer. Of particular interest is the dynamic control problem of how the repairman should allocate his/her capacity to the machines at any point in time so that the long-term average (weighted) sum of the queue lengths of the first-layer queues is minimized. Since the optimal policy for the repairman cannot be found analytically due to the correlations in the queue lengths, a near-optimal policy is proposed. This is obtained by combining intuition and results from queueing theory with techniques from Markov decision theory. Specifically, the relative value functions for several policies for which the model can be decomposed in less complicated subsystems are studied, and the results are combined with the classic one-step policy improvement algorithm. The resulting policy is easy to apply, is scalable in the number of machines, and is shown to be highly accurate over a wide range of parameter settings.