How to combine WEKA classifiers

Question

I need to utilize two different classifier to get best classification results. Since, it seems that they complement each other (not sure I am not expert btw). ROC characteristics are given below (testing scheme is 10-fold cross validation):

MetaCost [0 8; 1 0] Alternating Decision Tree (ADTree)

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.973     0.62       0.119     0.973     0.212      0.696    YES
                 0.38      0.027      0.994     0.38      0.55       0.696    NO
Weighted Avg.    0.427     0.074      0.925     0.427     0.523      0.696

MetaCost [0 8; 1 0] Logistic

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.604     0.161      0.245     0.604     0.348      0.771    YES
                 0.839     0.396      0.961     0.839     0.896      0.769    NO
Weighted Avg.    0.821     0.377      0.904     0.821     0.853      0.769

I have tried Voting but could not get desired results. Therefore, it is time to seek for expert help. Could you please advise me a solution, if possible?

Thanks in advance. Also, as a reminder I am not an expert.

EDIT: Best I can get:

Vote combines the probability distributions of these base learners:
    weka.classifiers.meta.AdaBoostM1 -P 100 -S 1 -I 10 -W weka.classifiers.meta.MetaCost -- -cost-matrix "[0.0 8.0; 1.0 0.0]" -I 10 -P 100 -S 1 -W weka.classifiers.trees.ADTree -- -B 10 -E -3
    weka.classifiers.meta.AdaBoostM1 -P 100 -S 1 -I 10 -W weka.classifiers.meta.MetaCost -- -cost-matrix "[0.0 8.0; 1.0 0.0]" -I 10 -P 100 -S 1 -W weka.classifiers.functions.Logistic -- -R 1.0E-8 -M -1
using the 'Product of Probabilities' combination rule 

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.706     0.204      0.231     0.706     0.348      0.825    YES
                 0.796     0.294      0.969     0.796     0.874      0.825    NO
Weighted Avg.    0.789     0.287      0.91      0.789     0.832      0.825

Why these two? Ensembles of several classifiers even of the same type are often better than any single one. — Michael R. Chernick, May 21 '12 at 14:50
@Michael Chernick, Since, Logistic seems better classifier for YES, whereas ADTree seems better for NO. I have tried Boosting logistic, it did not worked either. — baris.aydinoz, May 21 '12 at 14:53
Have you ever used Random Forests? If ensemble averages don't work why would combining these two be promising? — Michael R. Chernick, May 21 '12 at 17:03
@Michael Chernick, yes, but there must be a way to combine these to in a correct manner. — baris.aydinoz, May 21 '12 at 17:28
Possible duplicate of How to combine weak classfiers to get a strong one? — kjetil b halvorsen, Aug 14 '18 at 22:04

How to combine WEKA classifiers

0 Answers0