Markov Games with Several Ergodic Classes

Abstract

We consider Markov games of the general form characterized by the property that, for all stationary strategies of players, the set of game states is partitioned into several ergodic sets and a transient set, which may vary depending on the strategies of players. As a criterion, we choose the mean payoff of the first player per unit time. It is proved that the general Markov game with a finite set of states and decisions of both players has a value, and both players have ε-optimal stationary strategies. The correctness of this statement is demonstrated on the well-known Blackwell's example (“Big Match”).