Probability, Statistics, and Estimation Theory Assignment

This document presents solutions to a statistics and probability assignment. The solutions cover a range of topics including calculating marginal probability mass functions, variance, and expectations. The assignment explores concepts like unbiased estimators, mean squared error (MSE), and conditional expectation. Problems involve the analysis of random variables, regression analysis, and the application of statistical principles to solve complex problems. The solutions are detailed, explaining each step to facilitate understanding of the concepts. Furthermore, the assignment delves into the properties of estimators, including consistency and bias, and examines the relationships between different statistical measures. Desklib provides this and similar resources to help students excel in their studies.

Solution
Q1a)
E[n-1∑
i=1
n
XiYi ]
Marginal PME = {x+ y 0 ≤ ≤ x ≤≤ 1 ,0 ≤ y ≤1
0 otherwise 0
n = 4
∫
0
1
∫
0
1
1
4 ( x + y ) dxdy
= ¼ ∫
0
1
( xy + y2
2 |y = oy= 1)dx = ¼ *∫
0
1
( x + 1
2 ) dx
= ¼*[ x2
2 +1
2 x ¿01
= ¼[1/2 + ½ ] = ¼
b) Variance E[n-1∑
i=1
n
XiYi ]
= E[n-1∑
i=1
n
XiYi ]2 – (E[n-1∑
i=1
n
XiYi ])2
E[n-1∑
i=1
n
XiYi ]2 = ∫
0
1
∫
0
1
( 1
4 ( x+ y ) )
2
dxdy
= 1
16 ∫
0
1
[x2 y+x y2+ y3 ]01dx
= 1/16[ ∫
0
1
( x2 + x +1 ) dx= 1
16 [ x3
3 + x2
2 + x ]01 =
11
6 ∗1
16 = 11
96
Variance = 11/96 – 1/16 = 5/96
c) E[n-1∑
i=1
n
XiYi ]2 = ∫
0
1
∫
0
1
( 1
4 ( x+ y ) )
2
dxdy

= 1
16 ∫
0
1
[x2 y+x y2+ y3 ]01dx
= 1/16[ ∫
0
1
( x2 + x +1 ) dx= 1
16 [ x3
3 + x2
2 + x ]01
=
11
6 ∗1
16 = 11
96
d)
Let Xi = (1, Xi)’ for all i.
Find E[n-1∑
i=1
n
~X iYi
Marginal PME = {(1+x )+ y 0 ≤ ≤ x ≤≤ 1 , 0≤ y ≤1
0 otherwise 0
n = 4
∫
0
1
∫
0
1
1
4 ( 1+ x + y ) dxdy
= ¼ ∫
0
1
( y + xy + y2
2 |y = oy= 1)dx = ¼ *∫
0
1
(x +3
2 )dx
= ¼*[ x2
2 +3
2 x ¿01
= ¼[1/2 + 3/2 ] = ½
e) Variance E[n-1∑
i=1
n
~X iYi
= E[n-1∑
i=1
n
~X iYi ]2 – (E[n-1∑
i=1
n
~X iYi])2

E[n-1∑
i=1
n
~X iYi ]2 = ∫
0
1
∫
0
1
( 1
4 ( 1+ x+ y ) )
2
dxdy
= 1
16 ∫
0
1
[1+2 x+ 2
3 + x+ x2 + 1
3 ]01dx
= 1/16[ ∫
0
1
( 1+2 x + 2
3 + x+ x2 + 1
3 ) dx= 1
16 [ x +x2+ 2
3 x+ x2
2 + x3
3 + 1
3 x ]01 = 1
16 [1+1+ 2
3 + 1
2 + 1
3 + 1
3 ]
=23/96
Q2)
Let Y1, Y2, … Yn be i.i.d
E(E( ^Y ) = E (Y 1+… Yn )
n = [ E ( Y 1 ) +… . E ( Yn ) ]
n =μ2
^Y is an unbiased estimator
Therefore, the MSE of ^Y is
MSE ^μ2 = E( ( ^Y −μ 2 ) 2
=variance ( μ 2 ) = σ 2
n
b) If the estimator equals the parameter then it will be an unbiased estimator.
c)
^μ2 = 1/n∑
i=1
n
( Yi− ^Y ) 2
= n−1
n s2
E( ^μ22) = E( n−1
n S2
)=n−1
n μ2
The variance ^μ2 is calculated by
Variance ( ^μ2 ¿ = var ( n−1
n S2
)= ( n−1 )2
n2 var ( S2 )
=
( n−1 ) 2
n2 ∗2u4
n−1 = 2 ( n−1 ) u4
n2

d)
^μ2 = 1/n∑
i=1
n
( Yi− ^Y ) 2
= n−1
n s2
E( ^μ22) = E( c n−1
n S2
)=c n−1
n μ2
The variance ^μ2 is calculated by
Variance ( ^μ2 ¿ = var ( nc n−1
n S2
)=nc ( n−1 ) 2
n2 var ( S2 )
= nc
( n−1 )2
n2 ∗2 u4
n−1 =2 nc ( n−1 ) u4
n2
e) Therefore if ^μ2 (c ) the value of c > 1
E( ^μ2−μ2 ¿2 = var( ^μ2 ¿+Bia s2
= 2 ( n−1 ) μ2
n2 +( n−1
n μ2−μ2)
= 2(n−1)μ4
n2
f) c = 1/n
2∗1
n ∗n∗¿ ¿
= 2 ( n−1 ) u4
n2
g) on average ^μ2 will be closer to μ than S2 if MSE is used as a measure, however ^μ2 is biased
and will on average under estimate μ2.
Q3)

Let X1, X2,…Xn be i.i.d from N( μ , σ2 ¿ then ^X is an unbiased estimator of
μ∧S2 is an unbiased estimator for σ2
E( ^X ) = E( X 1+ …+ Xn
n ¿= E ( X 1 ) +…+ E( Xn)
n =μ
Then
^X is an unbiased estimator.
Q4)
a) No, there will be equal to each other
b)
E(g(X)|Y, Z) = E(X) + E(Y) = = E(g(X)|Y)
E(X+Y) = ∑
x
∑
y
( x+ y ) P( X =x , Y = y)
= ∑
x
∑
y
P ( X=x , Y = y )+∑
x
∑
y
yP( X=x , Y = y )
= ∑
x
x ∑
y
P ( X=x ,Y = y ) +¿ ¿ ∑
y
y ∑
x
P ( X =x , Y = y )

= ∑
x
xP ( X=x ) +∑
y
yP(Y = y)
= E(X) + E(Y)
c)
Cov(Y1, Y2) = E{(Y1 – EY1)(Y2 – EY2)} = E{(∑
i=1
n
ai (xi−EXi)¿ ¿
= ∑
i=1
n
aibiE ( Xi−EXi )2=¿ ∑
i=1
n
aibiVar( Xi) ¿
Cov *X, - ^X ^X ¿=cov ¿
= [1 – n-1)n-1 – (n – 1)n-2]Var(X) = 0
d)
E( ^X 1− ^X 2 ¿=E( ^X 1)−E( ^X 2)=μ 1−μ 2
Var[X + Y] = Var(X) + Var(Y) + 2cov(X,Y)
If X and Y are independent then cov(X, Y) = 0
Var( ^X 1− ^X 2 ¿=Var ( ^X 1)+Var ( ^X 2)=σ 12 /n1+ σ 22 /n 2
Q5a)

The β0 should be independent and identically distributed with the mean and variance , which is
no the case on this model presentation
ϵ= β0 – μ, ϵ whould be independent which is not the case on this model presentation
b)
u = ( β1 - μ β 1)x + ( β0 - μ β 0)
^γ=armin N−1
∑
i =1
N
( r 12 −γ 1−γ 2 x 12 )2
The γ 1is consistent estimator of averageincome difference between urban∧suburban individual
and γ 1is consistent estimator average income difference between urban and rural individual
Q6)
a)
( ^β 0 ^β 1 ^β 2 ¿' = arg min
β
∑
i=1
N
( y 1−β 0−1 xβ 2 i )
2
For estimator β vector
`σ = N-1∑
i=1
N
( yi− ^β 0− ^β 1 x i¿ ^β 2 )2

b)
β=¿
β ( F ( . ) )= [∫ xx ' dFx ( x) ]−1
∫ xydFx , y ( x , y)
= ¿]-1n−1
∑
i=1
n
XiYi
c)
Y1 = β 0 + β 1 xi+ μi
^β 1=¿
^β 1=
β 0 [∑
i=1
n
xi ]
[∑
i=1
n
x i2
]+ β 1+
[∑
i=1
n
x iμi ]
∑
i=1
n
x i2
E[ ^β 1] =
β 0 [∑
i=1
n
xi ]
β 0 [∑
i=1
n
x2 i ]+ β 1
d)

^β = (X’QX)-1X’QY = (X’QX)-1X’(QX β +Qϵ ¿
Second matrix X’QX
N-1∑
i=1
N
X ' QTXi p
→
E ¿TXi)
The second sum goes in probability as N → ∞
Q7)
Let g(X1) = E[Y|X1 = x1]. Find g(x1) for x1 = 1 and x1 = 0
Let h(x1, x2) = E[Y|(X1, X2) = (x1, x2)]. Find the value of this function at the four points in its
domain
a) g(X1) = E[Y|X1 = x1]
marginal PMF = { x+ y 0 ≤ x 1 ≤1 , 0 ≤ y 1≤ 1
0 otherwise
= ∫
0
1
∫
0
1
( x + y ) dxdy
∫
0
1
∫
0
1
( x + y ) dxdy
= ∫
0
1
(xy + y2
2 |y = oy= 1)dx = ∫
0
1
(x + 1
2 )dx
= [ x2
2 + 1
2 x ¿01
= [1/2 + ½ ] = 1
b) = ∫
0
1
0.33 dx+∫
0
1
0.17 dx+∫
0
1
0.31 dx +∫
0
1
0.19 dx
= 1
c) Fx(x) = ∑
y
fxy( x , y)

Fy(x) = ∑
x
fxy( x , y)
f XY(1,1) = 0.33
f XY(0,1) = 0.31
f XY(1,0) = 0.11
f XY(0,0) = 0.19
f XY(1,1) = 0.33
f XY(1,1) = 0.33
f XY(1,1) = 0.33
f XY(1,1) = 0.33
f XY = 1
d) ∫
0
1
1 dx = 1
e) From this argument then E[Y/X1] = E{E[Y]X1, X2]|X1]
Q8)
If ^β 1= c 1
c 2 ^β 1 and ^β 0=c 1 ^β 0
^β 1=
∑
i =1
n
c 1 c 2(xi−x)( yi− y)
∑
i=1
n
c 22 ( xi−x )2
= c 1
c 2 ^β 1
^β 0=¿ = c 1 ^β 0
β=c 1 ^β 0+ c 1
c 2 ^β 1
b)

Var( ^β 0 ¿=var (u ) +var ( ^β 1)x2
Var( ^β 0 ¿=
σ2 n−1
∑
i=1
n
x i2
∑
i=1
n
( xi−x )2