How do I combine two predictors?

Question

I am trying to classify a data set with 2 boolean values.

I have two classifiers that may/may not be independent. The first one is 65% accurate, and the second one is 60% accurate.

Can I combine the two somehow to get a better classifier? Or will the second one just weaken my prediction?

Boosting is what you're looking for. – Ansari May 05 '13 at 19:32 — Ansari, May 05 '13 at 19:32

score 2 · Accepted Answer · answered May 05 '13 at 20:51

Some kind of ensembling technique rather, which combines the two. The whole idea behind them are to combine weak classifiers into one powerful classifier.

Edit: Boosting is used in combination with an ensembler to increase prediction power/accuracy.

score 1 · Answer 2 · answered Sep 12 '13 at 21:35

As you only have two classifiers there only few options that come to my mind:

min/max value
Logical operators: AND, OR, XOR, NOR and the like.

However my suggestion is twofold:

First, I will focus not on the decision but on the score associated to that decision. If you use scores (probabilities, likelihood) you may use other ways of combining information (geometric mean, harmonic mean and so on).
Secondly, in case you are not interested on this former option, I would put my attention on those cases where both classifier disagree, analyze which of the former options (logical operators, min/max value) produce better performance than single classifiers and finally create a decisor-maker based on all these combinations.

I wish it works.

How do I combine two predictors?

2 Answers2