Logistic regression introduction

 Logistic Regression:

Logistic regression is a classification algorithm used to assign observations to a discrete set of classes. Unlike linear regression which outputs continuous number values, logistic regression transforms its output using the logistic sigmoid function to return a probability value which can then be to two or more discrete classes.

Types of  Logistic regression:

•  Binary ( pass/fail )
•  Multi ( cats,dogs,sheep )
•  Ordinal ( Low, Medium, High )

Sigmoid Function ( Logistic Function ):

Logistic regression also uses a linear equation with independent predictors to predict a value. The predicted value can be any where between negative infinity ( -ve ) to positive infinity ( +ve ) , we need the output of the algorithm to be class variable, i.e 0-No, 1-Yes. Therefore we are squashing the output of the linear equation into a range of [ 0-1 ], for this we use sigmoid function.

Linear equation and sigmoid equation:

we take the output( z )  of the linear equation and give to the function g(x), which returns squashed value h, this ( h ) will be lie in the range 0 to 1.

• where "e "= Euler's number ~ 2.71828

Decision boundary:

 Prediction function returns a probability score between 0 and 1, in order to map this to discrete class ( True/False ),we select threshold value or tipping point above which will classify as one class (1) and below classify as another class (0).

p≥0.5, class=1

p<0.5, class0

For instance, if our threshold value was 0.5, and our prediction function returned 0.7 we should classify this observation as positive,if our prediction was 0.2 then we should classify as negative. For logistic regression with multiple classes we could select the class with the highest predicted probability.