A Perturbation and Reduction Based Algorithm

We present a new algorithm for solving discrete-time Markov chains where transition probabilities have several orders of magnitude. Our method combines the decomposition by perturbation and the reduction approaches. In the decomposition phase, we remove from the chain some transitions with very small probabilities. But we assume that the reduced graph is still strongly connected after the deletion. A transversal (a cut set of the directed cycles) is then obtained and the reduction method is used to compute the first approximation of the steady-state distribution. Further approximations may be computed iteratively to obtain the desirable accuracy. We prove the algorithm and present some examples.