How to Implement Stacked Generalization (Stacking) From Scratch With Python

Last Updated on August 13, 2019

Code a Stacking Ensemble From Scratch in Python, Step-by-Step.

Ensemble methods are an excellent way to improve predictive performance on your machine learning problems.

Stacked Generalization or stacking is an ensemble technique that uses a new model to learn how to best combine the predictions from two or more models trained on your dataset.

In this tutorial, you will discover how to implement stacking from scratch in Python.

After completing this tutorial, you will know:

• How to learn to combine the predictions from multiple models on a dataset.
• How to apply stacked generalization to a real-world predictive modeling problem.

Kick-start your project with my new book Machine Learning Algorithms From Scratch, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

• Update Jan/2017: Changed the calculation of fold_size in cross_validation_split() to always be an integer. Fixes issues with Python 3.
• Update Aug/2018: Tested and updated to work with Python 3.6.