2
3

I have five independent classifiers. I have a dataset, and run each classifiers using the same dataset,so I can get five different results. Now I want to combine the results from the five classifiers in order to improve the accuracy.

Which methods are better and which software can solve this kind of problem? Thank you!

asked Oct 25 '11 at 11:47

yunfeiyu's gravatar image

yunfeiyu
45235


2 Answers:

Here are some things to try:

  1. Do a simple vote of the classifiers
  2. Concatenate the outputs of the five classifiers and train a new (simple linear) classifier on these vectors of classifier outputs (something similar to this worked well for the Netflix prize)
  3. Depending on your exact situation, you may be able to apply the Bayesian Committee Machine (google that phrase for papers)
  4. See Chris Bishop's PRML book in section 14.5 for mixtures of logistic regression models and so-called "mixtures of experts"

answered Oct 25 '11 at 14:55

gdahl's gravatar image

gdahl ♦
341453559

Now I think Mixture of expert and stacked generalized can deal with this problem. Do you know which software have these functions?

(Oct 25 '11 at 19:14) yunfeiyu

At least with stacked generalization, you're probably going to need to implement it yourself. It's more of a method of how you feed information to the classifier than some blackbox algorithm.

(Oct 26 '11 at 01:24) Kirk Roberts
1

I'd also suggest doing gdahl's options in order - I've had plenty of success with simple voting classifiers, and for actual implementations its rare I need to go past that.

(Oct 26 '11 at 04:58) Robert Layton

Alas, yunfeiyu, I don't know of software for you, and as Kirk mentioned it may be hard to find such software.

(Nov 07 '11 at 02:01) gdahl ♦

I am also looking for an stacking implementation, preferably in Python. Unfortunately, It seems that only Weka has a stacking implementation in the meta package.

(Mar 23 '12 at 22:00) Xinfan Meng

Though I did not read it entirely, the book Pattern Classification Using Ensemble Methods seems to treat this subject in depth.

answered Nov 10 '11 at 09:17

Lucian%20Sasu's gravatar image

Lucian Sasu
513172634

edited Nov 11 '11 at 06:47

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.