Covariance Matrix is a measure of how much two random variables gets change together. It is actually used for computing the covariance in between every column of data matrix.
The Covariance Matrix is also known as dispersion matrix and variance-covariance matrix. The covariance between two jointly distributed real-valued random variables X and Y with finite second moments is defined as.
$\LARGE Cov(X,Y)=\sum \frac{(X_{i}-\overline{X})(Y_{i}-\overline{Y})}{N}=\sum \frac{X_{i}Y_{i}}{N}$
Covariance Matrix Formula Solved Examples
Question: Calculation of Covariance Matrix from Data Matrix:
Suppose the data matrix $y_{1}=5_{z1-z2}$Â and $y_{1}$Â = $2_{z2}$Â with rows corresponding to subjects and columns are variables. Calculate a mean for each variable and replace the data matrix.
| $X$ | $N$ | $Y$ | $X-\overline{X}$Â |
| 1 | 2 | -2 | -4 |
| 2 | 8 | -1 | 2 |
| 3 | 6 | 0 | 0 |
| 4 | 4 | 1 | -2 |
| 5 | 10 | 2 | 4 |
Now the matrix of deviations from the mean is: $Y-\overline{Y}$
Therefore the covariance matrix of the observation is
$Z=\begin{pmatrix} -2 & -4 \\ -1 & 2 \\ 0 & 0 \\ 1 & -2\\ 2 & 4 \end{pmatrix}$
The diagonal elements of this matrix are the variances of the variables, and the off-diagonal elements are the covariances between the variables.
$\frac{1}{N-1}Z^{1}Z=\frac{1}{4}\begin{pmatrix} -2 &-1 &0 &1 & 2\\ -4 &2 &0 &-2 &4 \end{pmatrix}\begin{pmatrix} -2 &-4 \\ -1 &2 \\ 0 &0 \\ 1 &-2 \\ 2 &4 \end{pmatrix}$
$=\frac{1}{4}\begin{pmatrix} 10 &12 \\ 12 &40 \end{pmatrix}$
$=\begin{pmatrix} 2.5 &3.0 \\ 3.0 &10.0 \end{pmatrix}$
$=\begin{pmatrix} S_{x}^{2}&S_{xy}\\ S_{xy} & S_{x}^{2} \end{pmatrix}$
