This Tutorial talks about basics of Linear regression by discussing in depth about the concept of Linearity and Which type of linearity is desirable.
In Linear Regression the term linear is understood in 2 ways -
Linear regression however always means linearity in parameters , irrespective of linearity in explanatory variables.
A linear regression for 2 variables is represented mathematically as ( u is the error term )-
Y = B1 + B2X + u Or
Y = B1 + B2X ² + u
Here the variable X can be non linear i.e X or X² and still we can consider this as a linear regression. However if our parameters are not linear i.e say the regression equation is
Y = B1² + B2²X + u
then this can not be said to represent a linear regression equation.
|Model linear in parameters?
|Model linear in variables?|
|Yes||Linear Model||Linear Model|
|No||Non Linear Model||Non Linear Model|
A function Y = f(x) is said to be linear in X if X appears with a power or index of 1 only. i.e the terms such as x2, Γx, and so on are excluded or if x is not multiplied or divided by any other variable.
Y is linearly related to X if the rate of change of Y with respect to X (dY/dX) is independent of the value of X.
A function is said to be linear in the parameter, say, B1, if B1 appears with a power of 1 only and is not multiplied or divided by any other parameter (for eg B1 x B2 , or B2 / B1)
To reiterate again - For purpose of Linear regression we are only concerned about linearity of parameters B1, B2 .... and not the actual variables X1, X2 ....
For Log(Yi) = Log(B1) + B2 Log(Xi) + u
B2 is Linear but B1 is non-linear but if we transform α = Log(B1) then the model
Log(Yi) = α + B2 Log(Xi) + u
is linear in α and B2 as parameters. Implying we can make the regression equation linear in parameters using a simple transformation
For other cases we may not have an easy way to transform parameters to their linear form and such equations are hence treated as intrinsically non-linear and are NOT modeled using linear regression
This tutorial was originally posted here.
Next in the series :
Reference : Based on Lectures by Dr. Manish Sinha. ( Associate Prof. SCMHRD )