TODO
<aside> 📖 A model, $f$ is defined as linear if it is linear in its parameters, $\bm \theta$. More specifically, for :
$$ f(\lambda\bm{\theta_1} + \bm{\theta_2}, \mathbf x) = \lambda f(\bm{\theta_1}, \mathbf x) + f(\bm{\theta_2}, \mathbf x) $$
</aside>
The linearity of our inputs is not our concern.
The support vector machine (SVM) is a linear classifier that can be viewed as an extension of the perceptron. It finds the maximum margin of a separating hyperplane
Left: two possible separating hyperplanes Right: max margin separating hyperplane
We want to answer: What is the best separating hyperplane?
SVM’s approach: One that maximizes the distance to the closest distance from both classes, i.e. the hyperplane with the maximum margin
<aside> 📖 We define a linear classifier, $\mathbf{w}^\top \mathbf{x} + b$ for the classes $\{0, 1\}$ with the following decision rule:
$$ \mathbb{I}\left(\mathbf{w}^\top\mathbf{x} + b > 0\right) $$
</aside>
<aside> 📖 Let the margin $\gamma \in \mathbb{R}$ be defined as the distance from the hyperplane to the closest point of both classes.
The margin of $\mathcal{H}$ with respect to $\mathcal{D}$ is defined as:
$$ \gamma(\mathbf{w}, b) = \min_{\mathbf{x} \in \mathcal{D}}\frac{\lvert \mathbf{w}^\top\mathbf{x} + b \rvert}{\lVert \mathbf{w} \rVert_2} $$
</aside>