Gaussian distributions are widely used to model the real world. For example, we can employ them to describe errors of measurements or phenomena under the assumptions of the central limit theorem
<aside> 📖 In the multivariate case, each random variable is distributed normally, and their joint distribution is also gaussian
$$ \mathbf{x} = \begin{bmatrix} X_1 \\ \vdots \\ X_n \end{bmatrix} \sim \mathcal{N}(\bm{\mu}, \mathbf{\Sigma}) $$
The probability density function changes to become a mapping, $f : \mathbb{R}^n \rightarrow \mathbb{R}$, with the following definition:
$$ f(\mathbf{x} | \bm{\mu}, \mathbf{\Sigma}) = \frac{\exp\left( -\frac{1}{2} (\mathbf{x} - \bm{\mu})^\top(\mathbf{x} - \bm{\mu})\right)}{\sqrt{(2 \pi)^k\det\mathbf{\Sigma}}} $$
</aside>
The mean vector $\bm{\mu}$, describes the expected values of the distribution, that is, each of its components describes along its corresponding dimension:
$$ \bm{\mu} = \begin{bmatrix} \mathbb{E}[X_1] \\ \vdots \\ \mathbb{E}[X_n] \end{bmatrix} $$
$\mathbf{\Sigma}$ models the variance along each dimension, and how each of the random variables are correlated. The diagonal of $\mathbf{\Sigma}$ consists of the variance of each random variable while off-diagonal elements describe the correlation between two different random variables:
$$ \begin{align*}\bm{\Sigma} &= \mathbb{E}\left[(\mathbf{x} - \mathbb{E}[\mathbf{x}])(\mathbf{x} - \mathbb{E}[\mathbf{x}])^{\top}\right]\\ &=\begin{bmatrix}\mathbb{V}[X_1] && \text{Cov}[X_1, X_2] && \dots && \text{Cov}[X_1, X_N]\\\text{Cov}[X_2, X_1] && \mathbb{V}[X_2] && \dots && \text{Cov}[X_2, X_N] \\ \vdots && \vdots && \ddots && \vdots \\ \text{Cov}[X_N, X_1] && \text{Cov}[X_N, X_2] && \dots && \mathbb{V}[X_N]\end{bmatrix}\end{align*} $$
Visually, the distribution is centered around $\bm{\mu}$ while the covariance describes the shape. Consider the 2D case:
A Visual Exploration of Gaussian Processes
Gaussian distributions have the nice algebraic property of being closed under the operations of conditioning and marginalization.
Marginalization can be seen as integrating along one of the dimensions of the Gaussian distribution, which is in line with the general definition of the marginal distribution. Conditioning also has a nice geometric interpretation — we can imagine it as making a cut through the multivariate distribution, yielding a new Gaussian distribution with fewer dimensions.
