Covariance Matrix is a measure of how much two random variables gets change together. It is actually used for computing the covariance in between every column of data matrix.
The Covariance Matrix is also known as dispersion matrix and variance-covariance matrix. The covariance between two jointly distributed real-valued random variables X and Y with finite second moments is defined as.
\(\begin{array}{l}\LARGE Cov(X,Y)=\sum \frac{(X_{i}-\overline{X})(Y_{i}-\overline{Y})}{N}=\sum \frac{x_{i}y_{i}}{N}\end{array} \)
Where,
N = Number of scores in each set of data
X = Mean of the N scores in the first data set
\(\begin{array}{l}X_{i}\end{array} \)
=Â
\(\begin{array}{l}i^{th}\end{array} \)
 raw score in the first set of scores
\(\begin{array}{l}x_{i}\end{array} \)
=
\(\begin{array}{l}i^{th}\end{array} \)
deviation score in the first set of scores
Y = Mean of the N scores in the second data set
\(\begin{array}{l}Y_{i}\end{array} \)
=Â
\(\begin{array}{l}i^{th}\end{array} \)
 raw score in the second set of scores
\(\begin{array}{l}y_{i}\end{array} \)
=
\(\begin{array}{l}i^{th}\end{array} \)
deviation score in the second set of scores
Cov(X, Y) = Covariance of corresponding scores in the two sets of data
Covariance Matrix Formula Solved Examples
Question: Calculation of Covariance Matrix from Data Matrix:
Suppose the data matrix \(\begin{array}{l}y_{1}=5_{z1-z2}\end{array} \)
 and \(\begin{array}{l}y_{1}\end{array} \)
 = \(\begin{array}{l}2_{z2}\end{array} \)
 with rows corresponding to subjects and columns are variables. Calculate a mean for each variable and replace the data matrix.
\(\begin{array}{l}X\end{array} \) |
\(\begin{array}{l}N\end{array} \) |
\(\begin{array}{l}Y\end{array} \) |
\(\begin{array}{l}X-\overline{X}\end{array} \) Â |
| 1 |
2 |
-2 |
-4 |
| 2 |
8 |
-1 |
2 |
| 3 |
6 |
0 |
0 |
| 4 |
4 |
1 |
-2 |
| 5 |
10 |
2 |
4 |
Now the matrix of deviations from the mean is:
\(\begin{array}{l}Y-\overline{Y}\end{array} \)
Therefore the covariance matrix of the observation is
\(\begin{array}{l}Z=\begin{pmatrix} -2 & -4 \\ -1 & 2 \\ 0 & 0 \\ 1 & -2\\ 2 & 4 \end{pmatrix}\end{array} \)
The diagonal elements of this matrix are the variances of the variables, and the off-diagonal elements are the covariances between the variables.
\(\begin{array}{l}\frac{1}{N-1}Z^{1}Z=\frac{1}{4}\begin{pmatrix} -2 &-1 &0 &1 & 2\\ -4 &2 &0 &-2 &4 \end{pmatrix}\begin{pmatrix} -2 &-4 \\ -1 &2 \\ 0 &0 \\ 1 &-2 \\ 2 &4 \end{pmatrix}\end{array} \)
\(\begin{array}{l}=\frac{1}{4}\begin{pmatrix} 10 &12 \\ 12 &40 \end{pmatrix}\end{array} \)
\(\begin{array}{l}=\begin{pmatrix} 2.5 &3.0 \\ 3.0 &10.0 \end{pmatrix}\end{array} \)
\(\begin{array}{l}=\begin{pmatrix} S_{x}^{2}&S_{xy}\\ S_{xy} & S_{x}^{2} \end{pmatrix}\end{array} \)
