Stacking is a meta-ensemble machine learning technique that trains a second-level machine learning model on the predictions from multiple machine learning models trained on the data.
Stacking works by:
- Training a variety of machine learning models on the dataset
- Generating predictions from each of the trained models
- Training a second-level machine learning model (a meta-learner) on the predictions from step #2
Stacking apparently produces more accurate results than voting/averaging of ensemble predictions.
References:
- https://h2o-release.s3.amazonaws.com/h2o/rel-ueno/2/docs-website/h2o-docs/data-science/stacked-ensembles.html
- http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/
- Leo Breiman. “Stacked Regressions.” Machine Learning, 24, 49-64 (1996)
- Mark J van der Laan, Eric C Polley, and Alan E Hubbard. “Super Learner.” Journal of the American Statistical Applications in Genetics and Molecular Biology. Volume 6, Issue 1. (September 2007).
