Classification Metrics

 Cost function:

We create cost function and minimize it so that we can develop on accurate model with minimum error. If we use the cost function of the linear regression in 'Logistic regression' then it would be no use, it would be end up being a non-convex function with many local minimums, in which it would be very difficult to minimize the cost value and find the global minimum. Hence, we use Gradient descent to minimize cost value.

Performance Metrics for Classification Problems:

It is the easiest way to measure the performance of a classification problem where the output can be of two or more types of classes. A confusion matrix is nothing but a table with two dimensions 'Actual' and 'Predicted'.

For our better understanding give a label to the predicted output.

• when a person having cancer : 1
• when a person having no cancer : 0

True Positive ( TP ):

True positive are the cases when the actual class of the data point was 1( True ) and predicted also 1 (True) 

• True positive is the case where a person is actually having cancer (1) and model will classifying his case as cancer (1)

True Negative ( TN ):

True negatives are the cases when the actual class of the data points was 0 (False) and the predicted is also 0 (False).

• The case where a person Not having cancer and the model classifying his case as Not cancer, comes under True negative.

False Positive ( FP ):

False positives are the cases when the actual class of the data point was 0 (False) and the predicted is 1(True). False is because the model has predicted incorrectly and positive because the class predicted was a positive (1).

• A person Not having cancer and the model classifying his case as cancer comes under False positives.

False Negative ( FN ):

False negatives are the cases when the actual class of data point was 1 ( True ) and the predicted is 0 (False ).

• A person having cancer and the model classifying his case as No cancer, comes under False negative.

Accuracy:

It is the most common performance metric for classification algorithms. It may be defined as the number of correct predictions made as a ratio of all predictions made.

Classification reports:

This report consists of the scores of Precision, Recall, F1 and Support.

Precision:

In a classification task, the precision for a class is the number of true positives divided by the total numbers of elements labeled as belonging to the positive class.

High  Precision means that an algorithm returned substantially more relevant results than irrelevant ones.

Recall:

Recall is defined as the number of true positives divided by the total number of elements that actually belong to the positive class.

High  Recall means that the algorithm returned most of the relevant results.

F1 score:

Suppose we have 100 credit card transactions, of which 97 are legit and 3 are fraud and let's say we come up a  model that predictions everything as fraud. 

Now, if we take arithmetic mean of both then it comes out to be nearly 51%. We shouldn't be giving such a moderate score to a terrible model since it's just predicting every transaction as fraud.

So, we need something more balanced than arithmetic mean and i.e harmonic mean.

H.M = 2xy/(x+y)

F1 score = 2 * Precision * Recall / ( Precision+ Recall )
               = 2*3*100/(3+100) =  5%

F1 score is 1 indicates perfect, and 0 indicates worst.