On discounted AR–AT semi-Markov games and its complementarity formulations

In this paper, we introduce a class of two-person finite discounted AR–AT (Additive Reward–Additive Transition) semi-Markov games (SMGs). We provide counterexamples to show that AR–AT and AR–AT–PT (Additive Reward–Additive Transition Probability and Time) SMGs do not satisfy the ordered field property. Some results on AR–AT–AITT (Additive Reward–Additive Transition and Action Independent Transition Time) and AR–AIT–ATT (Additive Reward–Action Independent Transition and Additive Transition Time) games are obtained in this paper. For the zero-sum games, we prove the ordered field property and the existence of pure stationary optimals for the players. Moreover, such games are formulated as a vertical linear complementarity problem (VLCP) and have been solved by Cottle-Dantzig’s algorithm under a mild assumption. We illustrate that the nonzero-sum case of such games do not necessarily have pure stationary equilibria. However, there exists a stationary equilibria which has at most two pure actions in each state for each player.