Table of Contents

Probabilistic Framework for ML

Here we set up the basic framework in which we view machine learning from a probabilistic perspective

We have a random vector $X = (X_1, ..., X_d)$ that are either unobserved or observed

<aside> 📖 probabilistic (generative) models relate all variables over a joint probability distribution $p_{\theta}(X_1, ..., X_d)$, parametrized by $\theta$

</aside>

Assume variables (input, label pairs) are generated by some ‘true’ distribution, denoted $p_{\star}(X)$

Objective

Suppose we have a true joint distribution $p_{\star}$, which generates our inputs.

$$ (X_1, ..., X_n) \sim p_{\star}(X) $$

We want to specify a set of distributions $\mathcal P$, where we choose $p \in \mathcal P$ approximates $p_{\star}$

**TLDR: “**learn” $p_{\star}(X)$ by choosing $\theta$ to best match this distribution ($p_{\theta}$ to models $p_{\star}$)

Learning the joint distribution is essentially learning everything. This is a very hard task

Probabilistic Perspective

Under this setup, we can think of common machine learning tasks in terms of random variables:

Input Data: $X$
Discrete Outputs (labels): $C$
Continuous Outputs: $Y$

Learning & Inference

Moreover, we can frame common ML problems in terms of probabilistic models.