ROW-CONTINUOUS FINITE MARKOV CHAINS : STRUCTURE AND ALGORITHMS

A variety of bivariate finite Markov chain models are employed in the performance analysis of computer and production systems. Many such chains have a skip-free structure in their row index and can be described as row-continuous. For such a row-continuous chain, systematic treatment of the states in a row as a probabilistic entity permits development of a rank reducing algorithmic procedure for calculating the ergodic probabilities of the states. The algorithmic procedure produces, as byproducts, dynamic information such as first passage times, ergodic exit times and ergodic sojourn times, shedding additional light in understanding structural properties of row-continuous Markov chains. Many motivating examples are provided and some numerical results are exhibited.