What is Support Vector Machines (SVM) ?

Support Vector Machines

A Support Vector Machine (SVM) is a classifier that is defined using a separating hyperplane between the classes.This hyperplane is the N-dimensional version of a line.

Given labeled training data and a binary classification problem, the SVM finds the optimal hyperplane that separates the training data into two classes. This can easily be extended to the problem with N classes.

Let's consider a two-dimensional case with two classes of points. Given that it's 2D, we only have to deal with points and lines in a 2D plane. This is easier to visualize than vectors and hyperplanes in a high-dimensional space. Of course, this is a simplified version of the SVM problem, but it is important to understand it and visualize it before we can apply it to high-dimensional data.

Consider the following figure:

There are two classes of points and we want to find the optimal hyperplane to separate the two classes. But how do we define optimal? In this picture, the solid line represents the best hyperplane.

You can draw many different lines to separate the two classes of points, but this line is the best separator, because it maximizes the distance of each point from the separating line. The points on the dotted lines are called Support Vectors. The perpendicular distance between the two dotted lines is called maximum margin.

What is Confusion Matrix in Machine Learning ?

Confusion matrix

A Confusion matrix is a figure or a table that is used to describe the performance of a classifier. It is usually extracted from a test dataset for which the ground truth is known.

We compare each class with every other class and see how many samples are misclassified.

Naive Bayes classifier

Naïve Bayes classifier

Naïve Bayes is a technique used to build classifiers using Bayes theorem.

Bayes theorem describes the probability of an event occurring based on different conditions that are related to this event.

We build a Naïve Bayes classifier by assigning class labels to problem instances. These problem instances are represented as vectors of feature values. The assumption here is that the value of any given feature is independent of the value of any other feature. This is called the independence assumption, which is the naïve part of a Naïve Bayes classifier.