Econometrics Methods and Applications Chapter 1 Linear Regression

Chapter 1
Linear Regression
1.1. What is Regression
Regression means: the study of the dependence of one variable (the dependent variable), on
one or more other variables (the explanatory variables), with estimating and/or predicting the
population. See this example:
1.2. Types of Data
Economic data sets come in a variety of types. Whereas some econometric methods can be
applied with little or no modification to many different kinds of data sets, the special features
of some data sets must be accounted for or should be exploited. We next describe the most
important data structures encountered in applied work.
1.2.1. Cross-sectional data
Each observation is a new individual with information at a point in time :
1 observation= information about 1 cross-sectional unit.
Cross-sectional units: individuals, households, firms, cities, states data taken at a
given point in time.
Typical assumption: units form a random sample from the whole population the
notion of independence .
Observation wage educ exper female married
1
Econometrics
Methods and Applications

1 3.10 11 2 1 0
2 3.24 12 22 1 1
525 11.56 16 5 0 0
526 3.5 14 5 1 0
If the data is not a random sample, we have a sample selection problem
1.2.2. Time series data
Observations on economic variables over time
stock prices, money supply, CPI, GDP, annual homicide rates, automobile sales
frequencies: daily, weekly, monthly, quarterly, annually Unlike cross-
sectional data, ordering is important here!
typically, observations cannot be considered independent across time
year T-bill inflation population
2004 4.95 2.6 260,660
2005 5.21 2.8 263,034
1.2.3. Panel (or Longitudinal) data
Panel data or longitudinal data[1][2] are multi-dimensional data involving measurements over
time. Panel data contain observations of multiple phenomena obtained over multiple time
periods for the same individuals
unit year popul murders unemp police
1 2008 293,700 5 6.3 358
1 2010 299,500 7 7.4 396
2 2008 53,450 2 7.2 51
2 2010 51,970 1 8.1 51
1.3. Statistics
1.3.1. Arithmetic Mean
To obtain the value of the arithmetic mean, calculate the sum of the data and divide the result
by the total number of data. X is the symbol of the arithmetic mean.
2

1.3.2. Variance and Standard Deviation The sample
variance is represented by:
The standard deviation is the square root of the variance. The standard deviation
is represented by:
The standard deviation measures how concentrated the data are around the mean; the
more concentrated, the smaller the standard deviation
1.3.3. The covariance
Let be a scatter plot, put: )
3
)

=
In the general case, this formula is written like this:
Properties
Var(a)=0; Var(aX) = a2 Var (X); Var(X+a) = Var(X)
Cov(X,X) = Var(X, X)
Cov(X,Y ) = Cov(Y,X) (Symmetry of covariance)
Cov(aX + b, Y ) = a·cov(X,Y ) (Linearity with respect to X)
Cov(X, aY + b) = a·cov(X,Y ) (Linearity with respect to Y)
Var(X+Y) = Var(X) + Var(Y) + 2 Cov (X, Y)
If X and Y are independent then Cov(X, Y) = 0
1.3.4.. Coefficient of linear correlation (Pearson coefficient)
1.4. Simple linear Regression
Simple linear regression is the most commonly used technique for determining how one
variable of interest (the response variable) is affected by changes in another variable. The
terms "response" and "explanatory" mean the same thing as "dependent" and "independent",
but the former terminology is preferred because the "independent" variable may actually be
interdependent with many other variables as well. Simple linear regression is used for three
main purposes:
To describe the linear dependence of one variable on another.
To predict values of one variable from values of another, for which more data are
available.
To correct for the linear dependence of one variable on another, in order to clarify other
features of its variability. Linear regression determines the bestfit line through a scatter
plot of data, such that the sum of squared residuals is minimized; equivalently, it
4

minimizes the error variance. The fit is "best" in precisely that sense: the sum of squared
errors is as small as possible. That is why it is also termed "Ordinary Least Squares"
regression. Model of the simple linear regression is given by:
Simple regression = regression with 2 variables: Yt 1Xt + t
Yt Xt
Dependent variable Independent variable
Explained variable Explanatory variable
Response variable Control variable
Predicted variable Predictor variable
Regressand Regressor
0: Intercept (is the estimate of the mean outcome when x = 0 and should always be
stated in terms of the actual variables of the study)
1: the slope coefficient (The interpretation of 1
explanatory variable increases by one unit. This should always be stated in terms of the
actual variables of the study).
t: specification error (difference between the true model and the specified model).
An important objective of regression analysis is to estimate the unknown parameters 0 and
1 in the regression model. This process is also called fitting the model to the data, the
parameters 0 and 1 are usually called regression coefficients. The slope 1 is the change in
the mean of the distribution of Y producing a unit change in X. The intercept 0 is the mean
of the distribution of the response variable Y when X=0.
We are actually going to derive the linear regression model in three very different ways
these three ways reflect three types of econometric questions : descriptive, causal and
forecasting
1.5. Descriptive Analysis
Estimate E[y|x] (called the population regression function). In other words, we
5
0

6
f in E[ y| x ] = f(x ):
The simplest model is E[ y| x ] = 0 + 1 x