Ensemble Learning intro

 Ensemble Learning:

Ensemble learning helps improve Machine learning results by combining several models. This approach allows the production of better predictive performance compared to a single model.

• Train the different models on the same data set.
• Let each model make its predictions.
• Meta model: Aggregates prediction of individual models 
• Final Prediction: more robust and less prone to errors.
• Best results: models are skillful in different ways.

In general terms, Suppose you want know about a movie review, you can't depends on one person's opinion. There may be chances of bias (i..e He might be fan of that star cast of a particular movie). So you have to ask some random people, collect their opinions, then aggregate your answers. In many cases you will find that this aggregated answer is better than one's opinion. Similarly, you will often get best predictions, than with the best individual predictor. This is known as Ensemble learning technique.

Visual Explanation:

Advantages: Improvement in predictive accuracy.

Dis advantages: It is difficult to understand an Ensemble learning.

Ensemble learning Types:

1. Stacking & Blending:

Stacking is a way of combining multiple models, that introduces the concept of Meta learner. It is less widely used than bagging and boosting. Stacking used to combine the models of different types.

Procedure is:

• Split the training set into two disjoint sets
• Train several base learners on the first part
• Test the base learners on the second part
• Using the predictions from above step as the inputs, and the correct responses as the outputs, train a higher level learner.

Example: Voting Classifier.

* Blending is technique where we can do weighted averaging of final result.

2.Sequential Ensemble learning (Boosting):

Boosting is a machine learning ensemble meta-algorithm for principally, reducing bias. A group machine learning algorithms that convert weak learner to stronger one.

Sequential ensemble methods, where the base learners are generated sequentially.

Example: Ada Boost, Stochastic Gradient boosting, XG boosting.

3. Parallel Ensemble Learning(Bagging):

Bagging is a machine learning ensemble meta-algorithm intended to improve the strength and accuracy of  machine learning algorithms used in classification and regression purpose. It additionally diminishes fluctuation of data (variance) and help from over fitting.

Parallel ensemble methods where the base learners are generated in parallel.

Example: Random Forest, Bagged Decision trees, etc,.