Working of Naive Bayes

How Naive Bayes classifier works ?

Let's understand the working of Naive Bayes through an example. Here,we have weather conditions and playing sports.Now you need to classify whether players will play or not, based on weather conditions.

First Approach ( In case of single feature ) :

1. Calculate the prior probability for given class labels.
2. Find likelihood probability with each attribute for each class 
3. Put these value in Bayes formula and calculate posterior probability.
4. See, which class has a higher probability, given the input belongs to the higher probability class.

For simplifying prior and posterior probability calculation you can use the two tables one is Frequency table and another is likelihood table. Both of these tables will helps you to calculate the prior and posterior probability. The frequency table contains the occurrence of labels for all features. There are two likelihood tables, table 1 is showing prior probabilities of labels and table 2 is showing the posterior probability.

Now suppose you want to calculate the probability of playing when the weather is overcast.

Probability of playing :

P(Yes/ Overcast) = P( Overcast/Yes ) * P(Yes)/P(overcast) ---------1

Calculate prior probabilities:

P(Overcast) = 4/14 = 0.29

P(Yes) = 9/14 = 0.64

Calculate posterior probabilities:

P( Overcast/Yes ) = 4/9 = 0.44

Put prior and posterior probabilities in equation --1

P(Yes/ Overcast) = 0.44*0.64 / 0.29 = 0.98 (Higher).

Probability of Not playing :

P(No/ Overcast) = P( Overcast/No ) * P(No)/P(overcast) ---------2

Calculate prior probabilities:

P(Overcast) = 4/14 = 0.29

P(No) = 5/14 = 0.36

Calculate posterior probabilities :

P( Overcast/No ) = 0/5 = 0

Put prior and posterior probabilities in equation --2

P(No/ Overcast) = 0*0.36/ 0.29 = 0

The probability of 'Yes' class is higher. So, you can determine weather is overcast than players will play the sport.

Second Approach ( in case of multiple features ):

1. Calculate prior probability for given class labels 
2. Calculate conditional probability with each attribute for each class
3. Multiply same class conditional probability 
4. Multiply prior probability with step 3 probability.

Now suppose you want to calculate the probability of playing when the weather is Overcast, and the temperature is mild.

Probability of playing:

P( Play= Yes/ weather=overcast, Temp=Mild) = 

                                                            P(weather=overcast,Temp=Mild /play=Yes) * P(play=Yes) -------1

P(weather =overcast,Temp=Mild/play=Yes) = P (overcast/Yes) P(Mild/Yes)----2

1.Calculate prior probabilities : 

   P (Yes) = 9/14 =0.64 

2.Calculate posterior probabilities :

 : P(overcast/Yes) =4/9 = 0.44

 : P(Mild/Yes) = 4/9 = 0.44

3.Put posterior probabilities in equation --2

   P(weather=overcast, Temp=Mild/Play=Yes)= 0.44*0.44 = 0.1936 (Higher)

4.Put prior and posterior probabilities in equation --1

   P(play=Yes/weather = overcast, Temp=Mild) = 0.1936*0.64 = 0.124

Probability of Not playing:

P( Play= No/ weather=overcast, Temp=Mild) = 

                                                            P(weather=overcast,Temp=Mild /play=No) * P(play=No) -------3

P(weather =overcast,Temp=Mild/play=No) = P (overcast/No) P(Mild/No)----4

1.Calculate prior probabilities :

   P (No) = 5/14 =0.36 

2.Calculate posterior probabilities :

 : P(overcast/No) =0/9 = 0

 : P(Mild/No) = 2/5 = 0.4

3.Put posterior probabilities in equation 4

   P(weather=overcast, Temp=Mild/Play=No)= 0*0.4 = 0

4.Put prior and posterior probabilities in equation --3

   P(play=Yes/weather = overcast, Temp=Mild) = 0*0.36 = 0

The probability of 'Yes' class is higher. So, you can determine weather is overcast than players will play the sport.