On a reward rate estimation for the finite irreducible continuous-time Markov chain

A continuous-time homogeneous irreducible Markov chain {X(t)}, t ϵ[0,∞) taking values on {1,...,k},k <∞ is considered. Matrix (λij) of the intensity of transition λij from state i to state j is known. A unit of the sojourn time in state i gives reward βi so the total reward during time Y(t)=ʃt0βX(s)ds. The reward rates {βi}. are not known and it is necessary to estimate them. For that purpose the following statistical data on r observations are at our disposal: (1) t, observation time; (2) i, initial state X(0); (3) j, final state X(t); and (4) y, acquired reward Y(t). Two methods are used for the estimation: the weighted least-squares method and the saddle-point method for the Laplace transformation of the reward. Simulation study illustrates the suggested approaches. © 2017 Grace Scientific Publishing, LLC.