Statistical Analysis: Comparing ANOVA and Linear Regression Techniques

This report provides a comparative analysis of Analysis of Variance (ANOVA) and Linear Regression, two fundamental statistical techniques used to understand the relationships between variables. The report begins by defining linear regression as a method to estimate the functional relationship between dependent and independent variables, highlighting its assumptions and applications in predicting a response variable based on predictor variables, such as predicting lung capacity based on smoking habits. Conversely, ANOVA is introduced as a method to determine significant differences between levels of an independent variable, comparing the means at each level and examining the within-group and between-group variations. The report then illustrates ANOVA's application through a case study assessing student interest in different areas of psychology. Furthermore, the report clarifies that both ANOVA and linear regression are facets of a generalized linear model (GLM), differing primarily in their scope of application and data nature. The conclusion emphasizes that while the two techniques share a common mathematical foundation, their utility and relevance are distinct, based on the type of data and research questions addressed.

Running head: COMPARISON OF ANOVA AND LINEAR REGRESSION
COMPARISON OF ANOVA AND LINEAR REGRESSION
Name of Student
Name of University
Author Note

1
COMPARISON OF ANOVA AND LINEAR REGRESSION
Concepts of Linear regression and analysis of variance or ANOVA are statistical
techniques of understanding the behaviour of one variable with respect to another one. Linear
regression relates the variation of a dependent variable to that of one or more independent
variables. Analysis of variance is employed to determine whether there is any significant
difference between two or more levels of one or more factors.
Consider a variable “Y” dependent on the variables “X1” and “X2”. Linear regression
attempts to estimate the functional relationship of “Y” with “X1” and “X2”. The basic assumption
of linear regression includes that the dependent or response variable follows a normal
distribution and the errors in predicted values of “Y” from the actual values all identically and
independently follow standard normal distribution. The method then estimates the regression
coefficients, “β1” and “β2” for the independent variables which explain the changes in values of
“Y” due to unit change in the corresponding independent variable. Linear regression is therefore
a tool to estimate the relationship as well as predict the response variable on the basis of one or
more independent or predictor variables (Lowry, 2014). Suppose it is of interest to explain and
predict the lung function of a smoker as quantified by the lung capacity, by the number of
packets of cigarettes smoked per year and the number of years of being a smoker. Then a linear
equation on the predictor average packets smoked per year and number of years being a smoker
is fitted on the basis of available data points. The fitted regression equation then depending upon
its “goodness of fit” can predict lung capacity on for given predictor values.
ANOVA according to Gliner et al. (2017) is, “A statistical test conducted when there is a
nominal independent variable with three or more levels and a scale/continuous dependent
variable.” The tool can be employed to determine whether there is an effect or significant
differences on the observed values of the independent variables, for the subjects on account of

2
COMPARISON OF ANOVA AND LINEAR REGRESSION
thedifferent levels. It is a comparison of the meansat each level. The mechanism behind the
comparison is however done on the basis of the deviation of estimated factor level mean from the
grand mean which is the between group variation and the deviation of the observations around
the factor level mean which is the within group variation. The within group variation accounts
for the inherent randomness of the data whereas the between group variation accounts for both
inherent randomness as well as any non-random variation due to some plausible external factors
which indicate the differences among the levels (Roberts& Russo, 2014). It is these to variations
which are compared to determine whether there is any significant external influence or not.
Gliner et al. (2017), gives as example of a scenario where ANOVA is appropriate by referring to
a study byStark-Wroblewski et al. (2006), assessing familiarity and interest of undergraduate
students in five specific areas of professional psychology. The five levels were set to be, criminal
profiling, clinical psychology, school psychology, counselling and forensic psychology. The
interest of the students were quantified by an ordinal scale ranging from 0 which corresponds to
no interest to 5 which corresponds to high interest. The response of the students were then
recorded and the difference among the different levels were determined by using a single factor
ANOVA method.
Mathematically, both ANOVA and linear regression are facets of a generalized linear
model (GLM). A GLM is an ANOVA when all its predictors are categorical with a numeric
response whereas it is linear regression when both response and sets of predictors are
numerical(Fox, 2015). The difference however lies in the utility of the two tools, that is, their
scope of application as made apparent from the aforementioned examples.

3
COMPARISON OF ANOVA AND LINEAR REGRESSION
Therefore, it can be said that although the two statistical techniques are derived from the
same mathematical model under different specifications, they differ in the scope of application
and relevance as well as the nature of the data

4
COMPARISON OF ANOVA AND LINEAR REGRESSION
References
Fox, J. (2015). Applied regression analysis and generalized linear models. Sage Publications.
Gliner, J. A., Morgan, G. A., & Leech, N. L. (2017). Research methods in applied settings: An
integrated approach to design and analysis. Routledge.
Lowry, R. (2014). Concepts and applications of inferential statistics.
Roberts, M., & Russo, R. (2014). A student's guide to analysis of variance. Routledge.