1

Let $X_n$ is Markov chain for $n\geq 0$ and state space is $E=\{1,2,3\}$. One-step transition probability matrix is $$ \left[ \begin{matrix} p_{11}& p_{12}& p_{13}\\ p_{21}& p_{22}& p_{23}\\ p_{31}& p_{32}& p_{33}\\ \end{matrix} \right] $$ how can I solve $P(X_n =1)$?

As far as I know, if state space is $E=\{1,2\}$ and the one-step transition probability matrix's size is $2\times2$, $i.e.$ $$ \left[ \begin{matrix} p_{11}& p_{12}\\ p_{21}& p_{22}\\ \end{matrix} \right] $$ we have $$ \begin{split} q_n&=P\left( X_n=1 \right) \\ &=P\left( X_n=1\left| X_{n-1}=1 \right. \right) P\left( X_{n-1}=1 \right) +P\left( X_n=1\left| X_{n-1}=2 \right. \right) P\left( X_{n-1}=2 \right) \\ &=p_{11}P\left( X_{n-1}=1 \right) +p_{21}P\left( X_{n-1}=2 \right) \\ &=p_{11}q_n+p_{21}\left( 1-q_n \right) \\ &=\left( p_{11}-p_{21} \right) q_n+p_{21} \end{split} $$ then, we have $$ q_n-\frac{p_{21}}{p_{12}+p_{21}}=\left( p_{11}-p_{21} \right) \left( q_n-\frac{p_{21}}{p_{12}+p_{21}} \right) $$ $i.e.$,$\{q_n-\frac{p_{21}}{p_{12}+p_{21}}\}$ is geometric sequence. Therefore, if we know $P(X_0=1)=p_1$ and $P(X_0=2)=p_2$, we can easily find that $$ \begin{split} P\left( X_n=1 \right) &=q_n \\ &=q_0\left( p_{11}-p_{21} \right) ^n+\frac{p_{21}}{p_{12}+p_{21}} \\ &=P\left( X_0=1 \right) \left( p_{11}-p_{21} \right) ^n+\frac{p_{21}}{p_{12}+p_{21}} \\ &=p_1\left( p_{11}-p_{21} \right) ^n+\frac{p_{21}}{p_{12}+p_{21}} \end{split} $$ However, when the one-step transition probability matrix's size is to $3\times 3$, I can not find the recursion formula as above, and I even don't know how to solve this question.

Zeta
  • 11
  • You perform an eigenanalysis of $P$. There is no general formula once the dimension of $P$ exceeds $2\times 2.$ (The answers are algebraic functions of roots of polynomials, implying there are (nasty) formulas for three- and four-state chains, but not for anything larger.) See examples on our site for details. Sometimes, when the components of $P$ have nice relationships, formulas can be found: see https://stats.stackexchange.com/a/87440/919 for one example with three states. – whuber Feb 19 '23 at 14:29

0 Answers0