Linear regression is the most basic and commonly used predictive analysis. One variable is considered to be an explanatory variable, and the other is considered to be a dependent variable. For example, a modeler might want to relate the weights of individuals to their heights using a linear regression model.
There are several linear regression analyses available to the researcher.
Simple linear regression
- One dependent variable (interval or ratio)
- One independent variable (interval or ratio or dichotomous)
Multiple linear regression
- One dependent variable (interval or ratio)
- Two or more independent variables (interval or ratio or dichotomous)
Logistic regression
- One dependent variable (binary)
- Two or more independent variable(s) (interval or ratio or dichotomous)
Ordinal regression
- One dependent variable (ordinal)
- One or more independent variable(s) (nominal or dichotomous)
Multinomial regression
- One dependent variable (nominal)
- One or more independent variable(s) (interval or ratio or dichotomous)
Discriminant analysis
- One dependent variable (nominal)
- One or more independent variable(s) (interval or ratio)
Formula for linear regression equation is given by:
\[\large y=a+bx\]
a and b are given by the following formulas:
\(\large a \left(intercept\right)=\frac{\sum y \sum x^{2} – \sum x \sum xy} {(\sum x^{2}) – (\sum x)^{2}}\)\(\large b\left(slope\right)=\frac{n\sum xy-\left(\sum x\right)\left(\sum y\right)}{n\sum x^{2}-\left(\sum x\right)^{2}}\)
Where,
x and y are two variables on the regression line.
b = Slope of the line.
a = y-intercept of the line.
x = Values of the first data set.
y = Values of the second data set.
Solved Examples
Question: Find linear regression equation for the following two sets of data:
| Â x | Â 2 | Â 4 | Â 6 | 8 |
| y | 3 | 7 | 5 | Â 10 |
Solution:
Construct the following table:
| x | y | x2 | xy |
| 2 | 3 | 4 | 6 |
| 4 | 7 | 16 | 28 |
| 6 | 5 | 36 | 30 |
| 8 | 10 | 64 | 80 |
| Â $\sum x$ = 20 | Â $\sum y$ = 25 | Â $\sum x^{2}$ = 120 | Â $\sum xy$ = 144 |
$b$ = $\frac{n\sum xy-(\sum x)(\sum y)}{n\sum x^{2}-(\sum x)^{2}}$
$b$ = $\frac{4 \times 144 – 20 \times 25}{4 \times 120 – 400}$
b = 0.95
\(a=\frac{\sum y \sum x^{2} – \sum x \sum xy} {n(\sum x^{2}) – (\sum x)^{2}}\)
\(a=\frac{25\times 120 – 20\times 144} {4(120) – 400}\)
a = 1.5
Linear regression is given by:
y = a + bx
y = 1.5 + 0.95Â x