Support Vector Machines
A Support Vector Machine (SVM) is a classifier
that is defined using a separating hyperplane between the classes.This hyperplane is the N-dimensional
version of a line.
Given labeled training data and a binary
classification problem, the SVM finds the optimal hyperplane that separates the training
data into two classes. This can easily be extended to the problem with N classes.
Let's consider a two-dimensional case with
two classes of points. Given that it's 2D, we only have to deal with points and lines in a 2D
plane. This is easier to visualize than vectors and hyperplanes in a high-dimensional space.
Of course, this is a simplified version of the SVM problem, but it is important to understand
it and visualize it before we can apply it to high-dimensional data.
There are two classes of points and we want to find
the optimal hyperplane to separate the two classes. But how do we define optimal? In this
picture, the solid line represents the best hyperplane.
You can draw many different lines to separate the two
classes of points, but this line is the best separator, because it maximizes
the distance of each point from the separating line. The points on the dotted
lines are called Support Vectors. The perpendicular distance between the two
dotted lines is called maximum margin.
