Equilibrium distribution of Markov chain

Question

The transition matrix is $$P =\begin{bmatrix} \frac12 & \frac12 & 0 & 0 \\ \frac12 & \frac12 & 0 & 0 \\ 0 & 0 & \frac13 & \frac23 \\ 0 & 0 & \frac13 & \frac23\end{bmatrix}$$

Now the question is how can I find all equilibrium distribution of this chain. I have got $2$ which are $(0.5, 0.5, 0, 0)$ and $(0, 0, \frac13, \frac23)$

In order to find the other equilibrium distribution I tried $wP = w$ and I got $5$ equation upon which solving them I got $w_1=w_2, w_2=w_2, w_4 = 2w_3$ and $w_3=w_3$, so I conclude that any probability vector of form $(a , a, b, 1-2a-b)(a,b \in \mathbb{R})$ can be equilibrium distribution but when I tried with few numbers for $a$ and $b$ it not giving me the same probability vector that I started with.

I can't see what I am doing wrong

score 16 · Answer 1 · answered Oct 22 '20 at 15:18

16

Due to the sum to one criterion and $w_4=2w_3,$ the $a$ and $b$ also has to be chosen to satisfy this condition:

$$1-2a-b=2b$$

I think you miss out this condition, that is knowing $a$ would completely determine $b$. Also, we need each component to be nonnegative.

To recover your first solution, you can let $a=\frac12, b=0$.

To get your second solution, you can let $a=0,b=\frac13 $.

The general solution is the convex hull of the solution for each of the classes.

$$\alpha \left(\frac12, \frac12, 0, 0\right) + (1-\alpha)\left( 0,0, \frac13, \frac23\right)$$

where $0 \le \alpha \le 1$.

answered Oct 22 '20 at 15:18

Siong Thye Goh

6,781

Can you please explain how you got the final convex solution? – bluelagoon Oct 22 '20 at 15:29
I know the theoretical result before hand. But if you wish to derive it, first, express $b$ in terms of $a$ and then you study the range of $a$ to make each component nonnegative. – Siong Thye Goh Oct 22 '20 at 15:36
I tried doing that and I got (a a (1-2a)/3 (2-4a)/3 ) where 0 ≤ a ≤ 1/2, is it correct? – bluelagoon Oct 22 '20 at 22:51
1

Let $a=\frac12 \theta$ and our solution is the same. Just use convex combination in the future, it's much faster :). – Siong Thye Goh Oct 23 '20 at 01:49
@user295357 your solution can be expressed in the form given in the answer with $\alpha=\frac{2}{5}$, $\frac{2}{5}(1/2,1/2,0,0)+\frac{3}{5}(0,0,1/3,2/3)$ – Tyberius Oct 23 '20 at 18:06
@Tyberius: Good point ! Thanks. – user295357 Oct 23 '20 at 18:09

score 10 · Answer 2 · answered Oct 22 '20 at 15:31

This Markov Chain is not irreducible and is therefore not ergodic. That is the reason why there is no unique equilibrium distribution. More specifically: nonergodicity entails that the equilibrium distribution depends on the distribution of the initial state $X_0$ of this chain. This chain has two irreducible classes, {1,2} consisting of states 1 and 2, and {3,4} consisting of states 3 and 4. In the answer of @Siong Thye Goh, the parameter $\alpha$ has a precise interpretation, namely: $$ \alpha = \text{Prob}[X_0 \in \{1,2\}] $$

I don't think that this answers the question. OP already knows the invariant distribution is not unique. — Federico Poloni, Oct 23 '20 at 11:12

user295357 · Answer 3 · 2020-10-24T18:38:32.677

We have 3 restricting conditions on $w_{1}$, $w_{2}$, $w_{3}$ and $w_{4}$: (1) $w_{1}=w_{2}$, (2) $w_{4}=2w_{3}$, and (3) $w_{1}+w_{2}+w_{3}+w_{4}=1$. The general solution of (1) is $[w_{1},w_{2}] = w_{1}\times[1,1]$. The general solution of (2) is $[w_{3},w_{4}] = w_{3}\times[1,2]$. Finally from (3), we have $w_{1}+w_{2}+w_{3}+w_{4}=2w_{1} + 3w_{3}=1$. This last equation,

$2w_{1} + 3w_{3}=1$,

gives the general solution. To satisfy this equation, obviously, $w_{1}\leq1/2$ and $w_{3}\leq1/3$. For example, $w_{1}=1/4$ and $w_{3}=1/6$.

Therefore, $w_{1}$ and $w_{3}$ can be expressed as $w_{1}=1/2\times \alpha$ and $w_{3}=1/3 \times \beta$ respectively, where $0\leq\alpha\leq1$ and $0\leq\beta \leq1$. Substituting to above equation, we get $\beta = 1-\alpha$.

Equilibrium distribution of Markov chain

3 Answers3