Stacking is a 'Model of Models' technique. rather than being fed raw feature data an AI 'Stack' is fed the predictions of several other AI models.
Hit '>Play' on the image above to learn more about 'Stacking', or how to use AI to combine multiple AI models
Stacking involves training a whole array of different AI models on the same base training data and then using Data Fusion & AI techniques to optimally combine the output of multiple models.
For classification tasks we could simply train multiple models then let them vote and take the majority vote as our best prediction. Frankly there is nothing wrong with this approach. But, as we shall see, we can get even better outcomes by using a further AI model to determine the best prediction.
Once we have trained multiple models we can get them to make predictions on data that they have not seen. Then we can feed these predictions into an aggregator AI model. This aggregator or 'Stack' can then be trained on the predictions of each base model.
While we can expect that our base models will make errors it is unlikely that they will all make the same error. Our Aggregator (or Stack) is given the opportunity to learn which models can compensate for errors in other models. The result is more consistent predictions with higher overall accuracy.
Click on '>Play' in the video above to gain further insights into how stacks are assembled and trained. There is also an illustration of a practical example using a seven layer stack on the now-familiar loan default example.
Links to AI and Machine Learning methods discussed in the video:
Decision Trees: https://www.lucidate.co.uk/forum/machine-learning/ml-2-decision-trees
Random Forests: https://www.lucidate.co.uk/forum/ensemble-techniques/ensembles-1-random-forests-pt-1
AdaBoost: https://www.lucidate.co.uk/forum/ensemble-techniques/ensembles-3-adaboost
K-Nearest Neighbours: https://www.lucidate.co.uk/forum/machine-learning/ml-4-k-nearest-neighbours-knn
Support Vector Machines: https://www.lucidate.co.uk/forum/machine-learning/ml-3-support-vector-machines
Neural Networks: https://www.lucidate.co.uk/blog/categories/ai-education
Logistic Regression: https://www.lucidate.co.uk/forum/machine-learning/ml-5-logistic-regression