Probability Assignment: Analyzing a Dice Game with Markov Chains

This assignment delves into the application of Markov chains in a probability context, specifically analyzing a game involving rolling a six-sided die. The solution determines whether the game's scoring process constitutes a Markov chain, explaining the dependency of each state on the preceding one. It further investigates the ergodicity of the process, arguing that the consistent behavior in point acquisition allows for averaging over time. The assignment also assesses time-homogeneity, demonstrating that the transition probabilities remain constant. The solution includes the derivation of the transition probability matrix and the initial state distribution vector. Finally, it calculates the probability of a player's score being at least four points after four rolls, providing a detailed breakdown of the probability calculations. This assignment showcases a deep understanding of stochastic processes and their application in analyzing probabilistic systems.

Probability
Consider a game of repeatedly rolling a fair six-sided die. The player's score is accumulated by
adding one point when the outcome is 3,4,5 0r 6 and adding 2 when the outcome is 1 or 2. The
player starts off with a score of 0. The evolution of the players’ score over time can be described
as a discrete-time stochastic process.
i. Is this process a Markkov chain? Explain why/why not.
The process above is a Markkov chain in that it describes a sequence of possible events
whereas the probality of every event is merely dependent on the attained state of the previously
occurred event. For example, when the die is rolled the score of the player in this process is
determined through the accumulation of the point acquired on outcomes of the thrown dice. In
that relationship, the outcome depends on the rolling and the score depends on the previous roll
for deciding the toatl score of the specific player.
ii. Is the process ergodic? Explain why ? why not.
We can as well consider the process to be ergodic in that it the approach to select either
of the two broad categories of the winning of the point follows a specified behavior. The two
broad option is that a player can gain 1 point on rolling and a die and get either an outcome of
3,4,5 0r 6 and get 2 points when the outcome is 1 or 2. Both process to earn the player a score
follows a similar behaviour which can be averaged over time in regard to the averaged outcome
over the space for this process or system of rolling a die in its phase space. The process therefore
qualifies to be ergodic as it can be represented mathematically through the application of a
reasonable large selection of options to come up with the winner amongst the group of the
players present in the game.
iii. Is the process time-homogeneous ? Eplain why/why not?

The process above is time-homogenous. Homogenous means that the transition
probability from state i to state j does not change over time, that is, it remains constant. In our
case, from t to t+1, the probability of Xt+1=x is completely determined by our knowledge
of Xt=y, since Xt+1=y or Xt+1=t (if a six was thrown at time t, and this will depend only on the
probability of getting a given option, ie.1,2,3,4,5 or 6(at the t-th throw of the die), which we are
assuming constant over time. The process therefore qualifies to be time-homogeneous.
iv. Write down the transition probability matrix p for the process.
For i,j ≥ 0, P is given by
Pij = {10((1/6+2/6)+(3/6+4/6+5/6))j-1 j=1-i, i≥1 i=0 , otherwise.
The exponent j−1 would only make sense if the middle condition was amended
from i=0 to i=0,j≥1.
v. Write down the initial state distribution vector ∏(0) for the process.
μX(n)=P(X=n)= 5n−1 /6n
vi. Using the recursion ∏(n) = ∏(n-1) P, calculate the probability that the player’s score is at least 4
points, having rolled the die 4 times.
P(X=4)=(1/2)4
P(X=5)=(1/3)4
P(X=6)=(1/6) 4
P(X=1) = P(X>0) −P(X>1) =1− (5/6)4 = 1−625/1296 = 671/1296
P(X=1)=P(X>0)−P(X>1) =1−(56)4=1−625/1296=671/1296
P(X=2)=P(X>1)−P(X>2)=(5/6)4−(4/6)4=625/1296−256/1296=369/1296
P(X=3)=P(X>2)−P(X>3)=(4/6)4−(3/6)4=256/1296−81/1296=175/1296
P(X=4)=P(X>3)−P(X>4)=(3/6)4−(2/6)4=81/1296−16/1296=65/1296

P(X=5)=P(X>4)−P(X>5)=(2/6)4−(1/6)4=16/1296−1/1296=15/1296
P(X=6)=P(X>5)−P(X>6)=(1/6)4−0=1/1296
P(X=4)+P(X=5)+P(X=6)=(1/2)4
= 1/16