For example, I trained two models: one with SVM and one with KNN.
Final Prediction = 0.4*KNN + 0.6*SVM
Is this considered blending?
Asked
Active
Viewed 93 times
1
-
You can call it ensemble model. – Michael M Jul 29 '14 at 13:39
-
But there is no official term for this? Is this unheard of? – Jul 29 '14 at 13:48
1 Answers
2
Yes, you can call that blending. You could also call it a weighted average of two models.
Zach
- 23,766
-
1A side note: Why isn't this technique used widely? Is it because it is hard to find the appropriate weights? – Jul 29 '14 at 14:04
-
@user1008537 why do you think it's any good? You lose any theoretical properties of all approaches you're blending and end up with a droopy mess. – Marc Claesen Jul 29 '14 at 14:32
-
@user1008537 It is widely used. One easy way to find appropriate weights is through cross-validation. You can do a regression on the held-out data to find the weights. This is called a "second-stage model" or "meta model" – Zach Jul 29 '14 at 15:22
-
@MarcClaesen If your primary modeling goal is predictive accuracy, theoretical properties of individual models is far less important. In this case blending is often an excellent approach: the strengths of one model can make up for the weaknesses of another. – Zach Jul 29 '14 at 15:24