Support vector Machine - Intro

 Support Vector Machine (SVM) :

Support vector machine is a supervised machine learning algorithm that performs classification tasks by constructing hyper planes of classifiers in a multidimensional space that separates the data into two different classes. SVM  supports both regression and classification tasks and can handle multiple continuous and categorical variables.

You can see that in below diagram, there are two different classes of data( Blue and Red). You can separate them with several separation line such as A,B,C, and D. But there must be single separating line of hyper plane that classifies the two classes best. So, how to choose the best classifier?

Good vs Bad Classifier ?

Data can be separated by a line or a simple hyper plane is known as linearly separable data. These hyper planes that can linearly separate the data are known as linear classifier.

If the training data is linearly separable, we can obtain two parallel hyper planes that separate the two classes of data, so that the distance between them is maximized. The region bounded by these two hyper planes is called  the 'margin' and the maximum margin hyper plane is the hyper plane that lies half way between them.

The data points that lie closest to the hyper planes are called 'Support Vectors'. They are the data points most difficult to classify, because they lie at the boundaries of the margin data acts a support to maximize the margins.

In this example we have two separation lines for the data, but one of them is good. It is very clear the the margin on the diagram of right side is larger. So the hyper plane on the right side maximizes the distance between nearest data points.

We need to remember that a classifier is not worth when two classes seem to run too close to each other and when we run this same classifier through test data, there are high chances of misclassification. So the objective is to maximizes the margin, minimizes the risk involved in it and thus generalized it well on test data.

If you look at the data points below, you can't use linear classifier. To solve this we use non-linear classifier. 

The non-linear classifier works well to classify them and more importantly without any misclassification.

SVM algorithm works well on classification problems in multi dimensional space. These multiples or 'n' dimensions nothing but number of features in the data set. Hyper plane is just like a thin paper sheet, that separates two classes in a similar way as shown below.