Understanding Regression Terminology and Simple/Multiple Linear Regression

This article explains regression terminology like R2, Beta, Regression coefficient, Hierarchical regression, Multi-collinearity, Curvilinear, Homoscedasticity, Outlier, Residual and Simple/Multiple Linear Regression with solved examples. It also includes a summary output of a multiple regression analysis in Excel and estimation of performance for candidates based on the regression model.

Statistics
Name:
Institution:
2nd June 2018

Part 1 – Understanding Regression Terminology
 R2 versus Adjusted R2; the adjusted R2 is basically a modified version of R2 that has been
adjusted for the number of predictors in the model. The adjusted R2 increases if and only
if the new term improves the model more than would be expected by chance.
 Beta; this is a measure of volatility of a stock
 Regression coefficient; this is the constant ‘b’ in the regression equation that indicates the
change value in the dependent variable that corresponds to the unit change in the
independent variable.
 Hierarchical regression; this is a regression model that is used to show if variables of
interest explain a statistically significant amount of variance in the Dependent Variable
(DV) after accounting for all other variables.
 Multi-collinearity; refers to a phenomenon where one independent variable in a multiple
regression model can be linearly predicted from the other independent variables with a
substantial degree of accuracy.
 Curvilinear; refers to a smooth curve like a parabola or logarithmic curve.
 Homoscedasticity; refers to a situation where the variance around the regression line
(error term) is the same for all values of the predictor variable (X).
 Outlier; refers to an observation point that is way away or distant from other
observations.
 Residual; refers to the difference between the observed value of the dependent variable
(y) and the predicted value (ŷ).
Part 2 – Simple Linear Regression

The regression output is given below;
From the output we can say that the results are significant. The interview score (x1) is
statistically significant (p < 0.05).
The least square regression line is;
y=0.5122 x +1.4443
Estimation of Jeff’s performance. The interview score is 3.4 hence the performance would be;
y=0.5122∗3.4+1.4443=3.18578
Part 3 – Multiple Regression
The organization wants to use a combination of interview scores (x1), scores from a role playing
exercise (x2), and personality test scores (x3) to predict performance (y) in the employee
development program. An I/O psychologist collected data on the 32 employees who have already
participated in the program.
1. Run a multiple regression analysis in Excel.

Solution
Done in excel
2. Copy the summary output (Regression Statistics Table, ANOVA Table, and Coefficient
Table) into a Word document.
Solution
SUMMARY OUTPUT
Regression Statistics
Multiple R 0.678543
R Square 0.460421
Adjusted R
Square 0.402609
Standard Error 0.858196
Observations 32
ANOVA
df SS MS F
Significanc
e F
Regressio
n 3 17.59672 5.865573 7.964106 0.000541
Residual 28 20.62203 0.736501
Total 31 38.21875
Coefficient
s
Standard
Error t Stat P-value
Lower
95%
Upper
95%
Intercept 0.207852 0.725478 0.286504 0.776601 -1.27822 1.693926
Interview Score (x1) 0.335432 0.142997 2.345729 0.026302 0.042516 0.628347
Role Play Score (x2) 0.428836 0.196697 2.180183 0.037808 0.02592 0.831752
Personality Test Score
(x3) 0.013037 0.01412 0.923304 0.363737 -0.01589 0.041961
3. Are the results significant? Explain your response.
Solution

Yes the overall results are significant. The p-value for the F-Statistics if 0.000 (a value
less than 5% level of significance), we therefore reject the null hypothesis and conclude
that the model is different from zero hence the overall results are significant.
4. Should all predictors be included in the least squares regression line? Should any
predictors be excluded? Why or why not?
Solution
Not all the predictors should be included in the least squares regression line. This is
because some of the predictors such as Personality Test Score (x3) is insignificant (p > 0.05)
5. Use the summary output to find the least square regression line: y = ax1 + ax2 + ax3 +b
Solution
y=0.3354 x1 +0.4288 x2+0.013 x3 +0.2079
6. Estimate the y (performance) for the following two candidates (show your work): (See
Part3)
Solution
Laura:
y=0.3354 ( 5 ) +0.4288 ( 4 ) +0.013 ( 45 ) +0.2079=4.1851
Performance for Laura is 4.1851
Garry:
y=0.3354 ( 4 ) +0.4288 ( 5 ) +0.013 ( 40 ) +0.2079=4 .2135
Performance for Garry is 4.2135
7. Based on these results which candidate would you select into the employee development
program?
Solution

Based on the above results I would choose Gary since he had the highest score.