Q. 9.7 - A First Course in Probability

Question

A transition probability matrix is said to be doubly

stochastic if

$\sum _{i=0}^M{P}_{ij}=1$

for all states j = 0, 1, ... , M. Show that such a Markov chain is ergodic, then

j = 1/(M + 1), j = 0, 1, ... , M.

Step-by-Step Solution

Answer

It is proved that  a Markov chain is ergodic, then ${\mathrm{\pi }}_j= \frac{1}{M+1}$ for $j=0,1,...,M$

Step 1: Given Information

We have given that the transition probability matrix is  doubly stochastic if

$\sum _{i=0}^M{P}_{ij}=1$

for all states $j=0,1,...,M$.

Step 2: Simplify

As the chain ergodic and the transition matrix is doubly stochastic, there exists a unique stationary distribution $\mathrm{\pi }$. Now, we just have to check that is that ${\mathrm{\pi }}_i=\frac{1}{\mathrm{m}+!}$ solution of the system of the equation $\mathrm{\pi }=\mathrm{\pi P}$ i.e., is it true

${\mathrm{\pi }}_j = \sum _{\mathrm{i}} {\mathrm{p}}_{\mathrm{ij}} {\mathrm{\pi }}_{\mathrm{j}}$${\mathrm{\pi }}_{j }= \sum _{\mathrm{i}}{\mathrm{p}}_{{\mathrm{i}}_{\mathrm{j}}} {\mathrm{\pi }}_{\mathrm{j}}$

but, we have$\sum _{i }pij=1$, which means

$\frac{1}{M + 1}= \frac{1}{M + 1} . \sum _{i}pij=\frac{1}{M+1} .1 = \frac{1}{M+1}$

So, we have proved that the stationary distribution is ${\mathrm{\pi }}_j= \frac{1}{\mathrm{m}+1}$.