Final Project: Markov Chain Analysis and Simulation (EE380)

This document presents a solution to a Markov Chain project, likely for an Electrical Engineering course (EE380), involving the simulation and analysis of a Markov Chain with a given one-step transition matrix. The solution includes MATLAB code to verify the stochastic nature of the matrix, simulate the chain for 10,000 steps from a specified initial state distribution, and restart the chain 20 times with randomly generated initial distributions. The analysis focuses on the final state probabilities, checking if the sum of probabilities approximates 1.0 and determining whether the initial distributions are stationary. The solution also provides the MATLAB code and output results of the experiment.

Running head: MARKOV CHAIN (FINAL PROJECT)
MARKOV CHAIN (FINAL PROJECT)
Name of the Student
Name of the University
Author Note

1
MARKOV CHAIN (FINAL PROJECT)
The Markov Chain with one-step transition matrix is given below.
P =
[ 0.25 0.35 0.15 0 0.25
0.15 0.2 0.4 0.1 0.15
0.1 0.35 0.05 0.3 0.2
0.55 0 0.25 0.15 0.05
0 0.6 0.25 0 0.15 ]
1. The above transition matrix is right stochastic if the sum of each row is 1 or it can be right
stochastic if the sum of each column is 1 or it can be said that the matrix is doubly stochastic
if both sum of row and column sums up to 1. Now, a MATLAB script is written to find the
sum of each row and column.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15];
for i=1:length(P)
scol(i) = sum(P(:,i));
srow(i) = sum(P(i,:));
end
srow
scol = scol'
Output:
sumprob
srow =

2
MARKOV CHAIN (FINAL PROJECT)
1 1 1 1 1
scol =
1.0500
1.5000
1.1000
0.5500
0.8000
Hence, as the row sum is 1 hence the state transition matrix is right stochastic only.
2. The nth step of Markov chain simulation is given by,
Pn=π0∗Pn
Where, Pn is the probability vector after nth step.
π0 = initial state distribution.
n = simulation number.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15]; % state transition matrix
pi0 = [3/15 2/15 4/15 5/15 1/15]; % initial state distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n); % final state probability matrix
row_sum = sum(fprob);

3
MARKOV CHAIN (FINAL PROJECT)
fprob
row_sum
Output:
EE380_Exp10_A
fprob =
0.1781 0.3050 0.2316 0.1176 0.1676
row_sum =
1.0000
Hence, it can be seen that after 10000 simulation the sum of probabilities in the row is equal
to 1. Hence, the final state transition matrix is also right stochastic.
3. Now, the final probability array is generated after 10000 simulations for 20 randomly
chosen initial distributions. The initial distributions are chosen from uniform distribution
having values between [0,1].
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15]; % state transition matrix
for i=1:20
pi0 = rand(1,length(P)) % initial state distribution chosen randomly from uniform distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n) % final state probability matrix
row_sum = sum(fprob)

4
MARKOV CHAIN (FINAL PROJECT)
end
Output:
EE380_Exp10_B
pi0 =
0.6557 0.0357 0.8491 0.9340 0.6787
fprob =
0.5617 0.9617 0.7303 0.3709 0.5286
row_sum =
3.1533
pi0 =
0.7577 0.7431 0.3922 0.6555 0.1712
fprob =
0.4845 0.8295 0.6299 0.3199 0.4559
row_sum =
2.7198
pi0 =
0.7060 0.0318 0.2769 0.0462 0.0971
fprob =
0.2063 0.3532 0.2682 0.1362 0.1941
row_sum =

5
MARKOV CHAIN (FINAL PROJECT)
1.1581
pi0 =
0.8235 0.6948 0.3171 0.9502 0.0344
fprob =
0.5024 0.8601 0.6532 0.3317 0.4727
row_sum =
2.8201
pi0 =
0.4387 0.3816 0.7655 0.7952 0.1869
fprob =
0.4574 0.7832 0.5948 0.3021 0.4305
row_sum =
2.5679
pi0 =
0.4898 0.4456 0.6463 0.7094 0.7547
fprob =
0.5426 0.9289 0.7054 0.3583 0.5106
row_sum =
3.0457
pi0 =

6
MARKOV CHAIN (FINAL PROJECT)
0.2760 0.6797 0.6551 0.1626 0.1190
fprob =
0.3371 0.5772 0.4383 0.2226 0.3172
row_sum =
1.8924
pi0 =
0.4984 0.9597 0.3404 0.5853 0.2238
fprob =
0.4645 0.7953 0.6039 0.3067 0.4371
row_sum =
2.6076
pi0 =
0.7513 0.2551 0.5060 0.6991 0.8909
fprob =
0.5526 0.9462 0.7185 0.3649 0.5200
row_sum =
3.1023
pi0 =
0.9593 0.5472 0.1386 0.1493 0.2575
fprob =

7
MARKOV CHAIN (FINAL PROJECT)
0.3655 0.6258 0.4753 0.2414 0.3440
row_sum =
2.0519
pi0 =
0.8407 0.2543 0.8143 0.2435 0.9293
fprob =
0.5490 0.9400 0.7138 0.3625 0.5167
row_sum =
3.0821
pi0 =
0.3500 0.1966 0.2511 0.6160 0.4733
fprob =
0.3361 0.5755 0.4371 0.2220 0.3163
row_sum =
1.8870
pi0 =
0.3517 0.8308 0.5853 0.5497 0.9172
fprob =
0.5762 0.9865 0.7492 0.3805 0.5422
row_sum =

8
MARKOV CHAIN (FINAL PROJECT)
3.2347
pi0 =
0.2858 0.7572 0.7537 0.3804 0.5678
fprob =
0.4890 0.8372 0.6358 0.3229 0.4602
row_sum =
2.7450
pi0 =
0.0759 0.0540 0.5308 0.7792 0.9340
fprob =
0.4229 0.7240 0.5498 0.2792 0.3979
row_sum =
2.3738
pi0 =
0.1299 0.5688 0.4694 0.0119 0.3371
fprob =
0.2703 0.4627 0.3514 0.1785 0.2543
row_sum =
1.5171

9
MARKOV CHAIN (FINAL PROJECT)
pi0 =
0.1622 0.7943 0.3112 0.5285 0.1656
fprob =
0.3495 0.5983 0.4544 0.2308 0.3289
row_sum =
1.9619
pi0 =
0.6020 0.2630 0.6541 0.6892 0.7482
fprob =
0.5267 0.9017 0.6847 0.3478 0.4956
row_sum =
2.9564
pi0 =
0.4505 0.0838 0.2290 0.9133 0.1524
fprob =
0.3258 0.5578 0.4236 0.2151 0.3066
row_sum =
1.8291

10
MARKOV CHAIN (FINAL PROJECT)
pi0 =
0.8258 0.5383 0.9961 0.0782 0.4427
fprob =
0.5132 0.8787 0.6673 0.3389 0.4830
row_sum =
2.8811
Hence, ti can be seen from the above simulations with 20 restarts that the sum of row
probabilities are not equal to 1. Only, the sum is closest to 1 (1.1581) when the initial
distribution is
p i(0)=[0.7060 0.0318 0.27690.0462 0.0971]
Hence, if the sum of probabilities in the initial distribution is equal to 1 then the final
probabilities after simulation of Markov chain will be equal to 1.
4.
Now, distribution is not stationary as the sum of the probabilities of initial distribution is not
equal to 1 and the equation for being stationary distribution is not satisfied. The equation for
stationary distribution is
π∗P=πT
Where P = state transition matrix.
π = initial state vector.
The equation for the given transition matrix can be written in the form.

11
MARKOV CHAIN (FINAL PROJECT)
[ p i1
(0) p i2
(0 ) p i3
(0) p i4
(0) p i5
(0)
]
[0.25 0.35 0.15 0 0.25
0.15 0.2 0.4 0.1 0.15
0.1 0.35 0.05 0.3 0.2
0.55 0 0.25 0.15 0.05
0 0.6 0.25 0 0.15 ]=
[ p i1
(0 )
p i2
(0 )
p i3
(0 )
p i4
(0 )
p i5
(0 )
]
Where,
p i1
(0),… p i5
(0) are 1 to 5 elements of initial state vector.