Covariance Matrix¶
Definition¶
The covariance matrix is usually denoted as \(\boldsymbol{\Sigma}\), and it is a two dimensional array, where each entry corresponds to the covariance between two random variables:
\[
\begin{align}
\Sigma_{ij} &= \text{Cov}[X_{i}, X_{j}] \\
&= \mathbb{E}[(X_{i} - \mathbb{E}[X_{i}])\ (X_{j} - \mathbb{E}[X_{j}])]
\end{align}
\]
Where our multivariate random variable is: \(\mathbf{X} = (X_{1}, X_{2}, \dots, X_{d})\).
Notice that the diagonal of the covariance matrix corresponds to the variance of each entry:
\[
\Sigma_{ii} = \text{Cov}[X_{i}, X_{j}] = \text{Var}[X_{i}]
\]