TY - JOUR
T1 - Convergence of Controlled Models for Continuous-Time Markov Decision Processes with Constrained Average Criteria
AU - Zhang , Wenzhao
AU - Xiong , Xianzhu
JO - Annals of Applied Mathematics
VL - 4
SP - 449
EP - 464
PY - 2020
DA - 2020/08
SN - 35
DO - http://doi.org/
UR - https://global-sci.org/intro/article_detail/aam/18090.html
KW - continuous-time Markov decision processes, optimal value, optimal policies, constrained average criteria, occupation measures.
AB - <p style="text-align: justify;">This paper attempts to study the convergence of optimal values and optimal policies of continuous-time Markov decision processes (CTMDP for short)
under the constrained average criteria. For a given original model $\mathcal{M}$<sub>$∞$</sub> of
CTMDP with denumerable states and a sequence {$\mathcal{M}$<sub style="white-space: normal;">$n$</sub>} of CTMDP with finite states, we give a new convergence condition to ensure that the optimal
values and optimal policies of {$\mathcal{M}$<sub style="white-space: normal;">$n$</sub>} converge to the optimal value and optimal
policy of $\mathcal{M}$<sub style="white-space: normal;">$∞$</sub> as the state space $S$<sub>$n$</sub>&nbsp;of $\mathcal{M}$<sub style="white-space: normal;">$n$</sub> converges to the state space $S$<sub>$∞$</sub>&nbsp;of $\mathcal{M}$<sub>$∞$</sub>, respectively. The transition rates and cost/reward functions of $\mathcal{M}$<sub>$∞$&nbsp;</sub>are allowed to be unbounded. Our approach can be viewed as a combination
method of linear program and Lagrange multipliers.</p>