Regression Analysis | Assignment-1

Question 4
(a) Estimation of regression coefficients by least squares.
(i) Using R we obtainer estimates shown in table 1.
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 11.8563752 1.1049463 10.730 1.34e-05 ***
x1 -0.8790123 0.1846072 -4.762 0.00206 **
x2 -0.0001172 0.0423181 -0.003 0.99787
---
Therefore,
β0=11.8564, β1=−0.8790 and β2=−0.0001
(ii) The estimated 95% confidence interval is
−1.315539 ≤ β1 ≤−0.4424856
(iii) Given ( n−p ) ^σ 2
σ 2 χn− p
2
P ( χ(n− p ,0.975)
2 < ( n− p ) ^σ2
σ2 < χ(n− p ,0.025)
2
)=0.95
P ( χ(n −p , 0.975)
2
( n−p ) ^σ2 < 1
σ 2 < χ(n− p , 0.025)
2
( n− p ) ^σ 2 )=0.95
P ( ( n−p ) ^σ2
χ(n −p , 0.025)
2 < σ2 < ( n− p ) ^σ 2
χ(n− p , 0.97 5)
2 )=0.95
Therefore, the 95% confidence interval is
( n− p ) ^σ 2
χ(n− p , 0.025)
2 < σ2< ( n− p ) ^σ2
χ(n− p ,0. 975 )
2
(iv) Test the null hypothesis that β2=0.
H0 : β2=0
Ha : β2 ≠ 0
The test statistic is a t statistic (t) defined by
t= β2
SE with df = 10-2 = 8.
α =0.05
Decision, reject null hypothesis if |t | is greater than t8 ,0.025=2.306
t=−0.0001
0.0423 =−0.00236
Since, |t |=0.00236 is less than t8 ,0.025=2.306, fail to reject the null hypothesis
and conclude that β2=0.
(v) Test the null hypothesis that β1+ 10 β2=0
H0 : β1+10 β2=0
Ha : β1+10 β2 ≠ 0
The test statistic is a t statistic (t) defined by

t= β1+10 β2
SE1+10 SE2
with df = 10-2 = 8.
α =0.05
Decision, reject null hypothesis if |t | is greater than t8 ,0.025=2.306
t= 11.8564−0.001
0.1846+0.423 =19.512
Since, |t |=19.512 is greater than t8 ,0.025=2.306, reject the null hypothesis and
conclude that β1+ 10 β2 ≠0.
(vi) x1 i is the best linear predictor of y, because the estimated slope coefficient is
statistically significant at 95% confidence level.
(b) Given the following
y1 j =β1 +e1 j
y2 j =β1 + β2 +e2 j
y3 j =β1+ β2 + β3+ e3 j
(i) Express this model in the form y=Zβ+ e ,
Z=
(1 0 0
1 1 0
1 1 1)
(ii) Show that
^β1= y1 , ^β2= y2− y1, and ^β3= y3− y1
Let’s start with
y1 j =β1 +e1 j make e1 j the subject
e1 j= y1 j−β1 square both sides and sum
∑
j=1
5
e1 j
2 =∑
j =1
5
( y1 j− β1 )2
∑
j=1
5
e1 j
2 =∑
j =1
5
( y1 j
2−2 β1 y1 j+ β1
2 )
Let L=∑
j=1
5
e1 j
2
L=∑
j=1
5
( y1 j
2−2 β1 y1 j + β1
2 )
L=∑
j=1
5
y1 j
2−2 β1 ∑
j=1
5
y1 j +5 β1
2 differentiate with respect to β1 and equate to
zero.
∂ L
∂ β1
=−2 ∑
j=1
5
y1 j +10 β1=0
^β1=
∑
j =1
5
y1 j
5 = y1
Next,

y2 j =β1 + β2 +e2 j make e1 j the subject
e2 j= y2 j− y1−β2 square both sides and sum
∑
j=1
5
e2 j
2 =∑
j =1
5
( y2 j− y1−β2 )2
∑
j=1
5
e2 j
2 =∑
j =1
5
( y2 j
2−2 ( y1−β2 ) y2 j+ ( y1−β2 )2
)
Let L=∑
j=1
5
e2 j
2
L=∑
j=1
5
( y2 j
2−2 ( y1− β2 ) y2 j + ( y1− β2 ) 2
)
L=∑
j=1
5
y2 j
2−2 ( y1−β2 )∑
j=1
5
y2 j +5 ( y1
2−2 β2 y1 + β2
2 ) differentiate with respect to
β2 and equate to zero.
∂ L
∂ β1
=2∑
j=1
5
y2 j−10 y1 +10 β2=0
^β2=
∑
j =1
5
y2 j
5 − y1= y2− y1
Finally,
y3 j =β1+ β2 + β3+ e3 j make e1 j the subject
e3 j= y3 j− y2− β3 square both sides and sum
∑
j=1
5
e3 j
2=∑
j=1
5
( y3 j − y2−β3 )2
∑
j=1
5
e3 j
2=∑
j=1
5
( y3 j
2−2 ( y2−β3 ) y3 j + ( y2−β3 )2
)
Let L=∑
j=1
5
e3 j
2
L=∑
j=1
5
( y3 j
2−2 ( y2 −β3 ) y3 j + ( y2−β3 )2
)
L=∑
j=1
5
y3 j
2−2 ( y2−β3 ) ∑
j =1
5
y3 j +5 ( y2
2−2 β3 y2 + β3
2 ) differentiate with respect to
β3 and equate to zero.
∂ L
∂ β1
=2∑
j=1
5
y3 j−10 y2 +10 β3=0
^β2=
∑
j =1
5
y3 j
5 − y2= y3 − y2
(iii) Using R we obtain the column means for the data and thus we have the
estimates as follows: