Markov Chain (Final Project)

Verified

Added on  2022/11/14

|12
|1023
|401
AI Summary
This article explains the concept of Markov Chain and its simulation using MATLAB. It also discusses the stationary distribution of Markov Chain.

Contribute Materials

Your contribution can guide someone’s learning journey. Share your documents today.
Document Page
Running head: MARKOV CHAIN (FINAL PROJECT)
MARKOV CHAIN (FINAL PROJECT)
Name of the Student
Name of the University
Author Note

Secure Best Marks with AI Grader

Need help grading? Try our AI Grader for instant feedback on your assignments.
Document Page
1
MARKOV CHAIN (FINAL PROJECT)
The Markov Chain with one-step transition matrix is given below.
P =
[ 0.25 0.35 0.15 0 0.25
0.15 0.2 0.4 0.1 0.15
0.1 0.35 0.05 0.3 0.2
0.55 0 0.25 0.15 0.05
0 0.6 0.25 0 0.15 ]
1. The above transition matrix is right stochastic if the sum of each row is 1 or it can be right
stochastic if the sum of each column is 1 or it can be said that the matrix is doubly stochastic
if both sum of row and column sums up to 1. Now, a MATLAB script is written to find the
sum of each row and column.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15];
for i=1:length(P)
scol(i) = sum(P(:,i));
srow(i) = sum(P(i,:));
end
srow
scol = scol'
Output:
sumprob
srow =
Document Page
2
MARKOV CHAIN (FINAL PROJECT)
1 1 1 1 1
scol =
1.0500
1.5000
1.1000
0.5500
0.8000
Hence, as the row sum is 1 hence the state transition matrix is right stochastic only.
2. The nth step of Markov chain simulation is given by,
Pn=π0Pn
Where, Pn is the probability vector after nth step.
π0 = initial state distribution.
n = simulation number.
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15]; % state transition matrix
pi0 = [3/15 2/15 4/15 5/15 1/15]; % initial state distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n); % final state probability matrix
row_sum = sum(fprob);
Document Page
3
MARKOV CHAIN (FINAL PROJECT)
fprob
row_sum
Output:
EE380_Exp10_A
fprob =
0.1781 0.3050 0.2316 0.1176 0.1676
row_sum =
1.0000
Hence, it can be seen that after 10000 simulation the sum of probabilities in the row is equal
to 1. Hence, the final state transition matrix is also right stochastic.
3. Now, the final probability array is generated after 10000 simulations for 20 randomly
chosen initial distributions. The initial distributions are chosen from uniform distribution
having values between [0,1].
MATLAB code:
P = [0.25 0.35 0.15 0 0.25;0.15 0.2 0.4 0.1 0.15;0.1 0.35 0.05 0.3 0.2;0.55 0 0.25 0.15 0.05;0
0.6 0.25 0 0.15]; % state transition matrix
for i=1:20
pi0 = rand(1,length(P)) % initial state distribution chosen randomly from uniform distribution
n = 10000; % number of steps is 10000
fprob = pi0*(P^n) % final state probability matrix
row_sum = sum(fprob)

Secure Best Marks with AI Grader

Need help grading? Try our AI Grader for instant feedback on your assignments.
Document Page
4
MARKOV CHAIN (FINAL PROJECT)
end
Output:
EE380_Exp10_B
pi0 =
0.6557 0.0357 0.8491 0.9340 0.6787
fprob =
0.5617 0.9617 0.7303 0.3709 0.5286
row_sum =
3.1533
pi0 =
0.7577 0.7431 0.3922 0.6555 0.1712
fprob =
0.4845 0.8295 0.6299 0.3199 0.4559
row_sum =
2.7198
pi0 =
0.7060 0.0318 0.2769 0.0462 0.0971
fprob =
0.2063 0.3532 0.2682 0.1362 0.1941
row_sum =
Document Page
5
MARKOV CHAIN (FINAL PROJECT)
1.1581
pi0 =
0.8235 0.6948 0.3171 0.9502 0.0344
fprob =
0.5024 0.8601 0.6532 0.3317 0.4727
row_sum =
2.8201
pi0 =
0.4387 0.3816 0.7655 0.7952 0.1869
fprob =
0.4574 0.7832 0.5948 0.3021 0.4305
row_sum =
2.5679
pi0 =
0.4898 0.4456 0.6463 0.7094 0.7547
fprob =
0.5426 0.9289 0.7054 0.3583 0.5106
row_sum =
3.0457
pi0 =
Document Page
6
MARKOV CHAIN (FINAL PROJECT)
0.2760 0.6797 0.6551 0.1626 0.1190
fprob =
0.3371 0.5772 0.4383 0.2226 0.3172
row_sum =
1.8924
pi0 =
0.4984 0.9597 0.3404 0.5853 0.2238
fprob =
0.4645 0.7953 0.6039 0.3067 0.4371
row_sum =
2.6076
pi0 =
0.7513 0.2551 0.5060 0.6991 0.8909
fprob =
0.5526 0.9462 0.7185 0.3649 0.5200
row_sum =
3.1023
pi0 =
0.9593 0.5472 0.1386 0.1493 0.2575
fprob =

Paraphrase This Document

Need a fresh take? Get an instant paraphrase of this document with our AI Paraphraser
Document Page
7
MARKOV CHAIN (FINAL PROJECT)
0.3655 0.6258 0.4753 0.2414 0.3440
row_sum =
2.0519
pi0 =
0.8407 0.2543 0.8143 0.2435 0.9293
fprob =
0.5490 0.9400 0.7138 0.3625 0.5167
row_sum =
3.0821
pi0 =
0.3500 0.1966 0.2511 0.6160 0.4733
fprob =
0.3361 0.5755 0.4371 0.2220 0.3163
row_sum =
1.8870
pi0 =
0.3517 0.8308 0.5853 0.5497 0.9172
fprob =
0.5762 0.9865 0.7492 0.3805 0.5422
row_sum =
Document Page
8
MARKOV CHAIN (FINAL PROJECT)
3.2347
pi0 =
0.2858 0.7572 0.7537 0.3804 0.5678
fprob =
0.4890 0.8372 0.6358 0.3229 0.4602
row_sum =
2.7450
pi0 =
0.0759 0.0540 0.5308 0.7792 0.9340
fprob =
0.4229 0.7240 0.5498 0.2792 0.3979
row_sum =
2.3738
pi0 =
0.1299 0.5688 0.4694 0.0119 0.3371
fprob =
0.2703 0.4627 0.3514 0.1785 0.2543
row_sum =
1.5171
Document Page
9
MARKOV CHAIN (FINAL PROJECT)
pi0 =
0.1622 0.7943 0.3112 0.5285 0.1656
fprob =
0.3495 0.5983 0.4544 0.2308 0.3289
row_sum =
1.9619
pi0 =
0.6020 0.2630 0.6541 0.6892 0.7482
fprob =
0.5267 0.9017 0.6847 0.3478 0.4956
row_sum =
2.9564
pi0 =
0.4505 0.0838 0.2290 0.9133 0.1524
fprob =
0.3258 0.5578 0.4236 0.2151 0.3066
row_sum =
1.8291

Secure Best Marks with AI Grader

Need help grading? Try our AI Grader for instant feedback on your assignments.
Document Page
10
MARKOV CHAIN (FINAL PROJECT)
pi0 =
0.8258 0.5383 0.9961 0.0782 0.4427
fprob =
0.5132 0.8787 0.6673 0.3389 0.4830
row_sum =
2.8811
Hence, ti can be seen from the above simulations with 20 restarts that the sum of row
probabilities are not equal to 1. Only, the sum is closest to 1 (1.1581) when the initial
distribution is
p i(0)=[0.7060 0.0318 0.27690.0462 0.0971]
Hence, if the sum of probabilities in the initial distribution is equal to 1 then the final
probabilities after simulation of Markov chain will be equal to 1.
4.
Now, distribution is not stationary as the sum of the probabilities of initial distribution is not
equal to 1 and the equation for being stationary distribution is not satisfied. The equation for
stationary distribution is
πP=πT
Where P = state transition matrix.
π = initial state vector.
The equation for the given transition matrix can be written in the form.
Document Page
11
MARKOV CHAIN (FINAL PROJECT)
[ p i1
(0) p i2
(0 ) p i3
(0) p i4
(0) p i5
(0)
]
[0.25 0.35 0.15 0 0.25
0.15 0.2 0.4 0.1 0.15
0.1 0.35 0.05 0.3 0.2
0.55 0 0.25 0.15 0.05
0 0.6 0.25 0 0.15 ]=
[ p i1
(0 )
p i2
(0 )
p i3
(0 )
p i4
(0 )
p i5
(0 )
]
Where,
p i1
(0),… p i5
(0) are 1 to 5 elements of initial state vector.
1 out of 12
circle_padding
hide_on_mobile
zoom_out_icon
[object Object]

Your All-in-One AI-Powered Toolkit for Academic Success.

Available 24*7 on WhatsApp / Email

[object Object]