Abstract
We prove that if a certain row of the transition probability matrix of a regular Markov chain is subtracted from the other rows of this matrix and then this row and the corresponding column are deleted, then the spectral radius of the matrix thus obtained is less than 1. We use this property of a regular Markov chain for the construction of an iterative process for the solution of the Howard system of equations, which appears in the course of investigation of controlled Markov chains with single ergodic class and, possibly, transient states.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
V. I. Romanovskii, Discrete Markov Chains [in Russian], Gostekhteoretizdat, Moscow (1949).
T. A. Sarymsakov, Foundations of the Theory of Markov Processes [in Russian], Fan, Tashkent (1988).
R. A. Howard, Dynamic Programming and Markov Processes, Wiley, New York (1960).
H. Mine and S. Osaki, Markovian Decision Processes [Russian translation], Nauka, Moscow (1977).
E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes and Their Applications [in Russian], Nauka, Moscow (1975).
V. I. Krylov, V. V. Bobkov, and P. I. Monastyrnyi, Computational Methods [in Russian], Nauka, Moscow (1976).
R. A. Horn and C. R. Johnson, Matrix Analysis [Russian translation], Mir, Moscow (1989).
J. G. Kemeny and J. L. Snell, Finite Markov Chains [Russian translation], Nauka, Moscow (1970).
C. Derman, “On sequential decisions and Markov chains,” Manag. Sci., 9, 16–24 (1962).
O. V. Viskov and A. N. Shiryaev, “On controls leading to optimal stationary modes,” Tr. Mat. Inst. Akad. Nauk SSSR, 71, 35–45 (1964).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ibragimov, A.A. On One Property of a Regular Markov Chain. Ukrainian Mathematical Journal 54, 570–576 (2002). https://doi.org/10.1023/A:1021079110162
Issue Date:
DOI: https://doi.org/10.1023/A:1021079110162