• Table of Contents

Using an infinite-dimensional gaussian to represent functions you want

• Infinite dimensional → represents any function over $\mathbb R^n$
  • Since it’s a probability distribution over functions, you can bayes it
• Covariance mat/kernel → design prior for desired function families/properties

Derivation

We are interested in a model of the form: $p(y | x)$, i.e. a predictive distribution

• Approach 1: Ordinary linear regression
• Approach 2: Bayesian linear regression

The main framing to keep in mind

<aside> 📖

A gaussian distribution with a mean vector $\bm \mu$, and a covariance matrix $\bm \Sigma$ defines a function $\mathbf f$:

$$ \mathbf f = (f_1, \dots, f_n) \sim \mathcal N(\mu, \bm \Sigma) $$

</aside>

• This already covers functions with a countable domain (cast $f_i$ to $f(i)$)
• Second, notice we can rank 1d sequences/signals by probability, and even sample them
• Some “function” samples from a multivariate gaussian

The name

GPs are a generalization of a multivariate gaussian distribution to infinitely many variables

<aside> 📖

A gaussian process is a collection of random variables, any finite number of which, have consistent Gaussian distributions. Alternatively, we can specify this by:

1. A mean function $m(\mathbf x)$
2. A covariance function, i.e. kernel $K(\mathbf x, \mathbf x')$

We denote this:

$$ f(\mathbf x) \sim \mathcal{GP}(m(\mathbf x), K(\mathbf x, \mathbf x')) $$

</aside>

Sources