This Tutorial talks about basics of Linear regression by discussing in depth about the concept of Linearity and Which type of linearity is desirable.

What is the meaning of the term Linear ?

In Linear Regression the term linear is understood in 2 ways -

1. Linearity in variables
2. Linearity in parameters

Linear regression however always means linearity in parameters , irrespective of linearity in explanatory variables.

A linear regression for 2 variables is represented mathematically as ( u is the error term )-

Y = B1 + B2X + u          Or

Y = B1 + B2X ² + u

Here the variable X can be non linear i.e X or X² and still we can consider this as a linear regression. However if our parameters are not linear i.e say the regression equation is

Y = B1² + B2²X + u

then this can not be said to represent a linear regression equation.

Linear Regression Models

Linearity in predictor variables - Xi

A function Y = f(x) is said to be linear in X if X appears with a power or index of 1 only. i.e the  terms such as x2, Γx, and so on are excluded or  if x is not multiplied or divided by any other variable.

Linearity in parameters - Bi

Y is linearly related to X if the rate of change of Y with respect to X (dY/dX) is independent of the value of X.

A function is said to be linear in the parameter, say, B1, if B1 appears with a power of 1 only and is not multiplied or divided by any other parameter (for eg B1 x B2 , or B2 / B1)

To reiterate again - For purpose of Linear regression we are only concerned about linearity of parameters B1, B2 .... and not the actual variables X1, X2 ....

Non Linear Models

• Some models may look non linear in the parameters but are inherently or intrinsically linear.
• This is because with suitable transformations they can be made linear in parameters.
• However, if these cannot be linearized, these are called intrinsically non linear regression models
• When we say ‘non linear regression model’ we mean that it is intrinsically non linear.

Example

For  Log(Yi) = Log(B1) + B2 Log(Xi) + u

B2 is Linear but B1 is non-linear but if we transform α = Log(B1) then the model

Log(Yi) = α + B2 Log(Xi) + u

is linear in α and B2 as parameters. Implying we can make the regression equation linear in parameters using a simple transformation

For other cases we may not have an easy way to transform parameters to their linear form and such equations are hence treated as intrinsically non-linear and are NOT modeled using linear regression

This tutorial was originally posted here. 

Next in the series :

• Tutorial : Linear Regression Construct
• R Tutorial : Basic 2 variable Linear Regression
• R Tutorial : Multiple Linear Regression
• R Tutorial : Residual Analysis for Regression
• R Tutorial : How to use Diagnostic Plots for Regression Models
• R Tutorial : How to interpret F Statistic in Regression Models

Reference : Based on Lectures by Dr. Manish Sinha. ( Associate Prof. SCMHRD )