|
I am currently working on a text classification problem. My tool box of choice is scikit-learn, because it lets me experiment quickly with different algorithms. Looking at the data it seems, that the documents fall into different clusters/categories (think text genres), which have different dynamics (different features are important, different class distribution, ...). This makes we think that a mixture of experts model might be a good idea. As I understand it, mixture of experts requires you to give different weights to each training example to train the expert. Unfortunately any off-the-shelf learning algorithm implementation I know of does not support this. So, is there a way I could validate this idea without spending too much time on it? I don't have practical experience with neural networks and my intuition says that neural networks would not do well on this dataset (too small and noisy). One option would be using Naive Bayes, which is quick to implement and doing ok on this dataset (but not as well as logistic regression/SVM) . Another would be duplicating training examples. |
I've not heard of many succesful usages of mixtures of experts for document classification. scikit-learn has many good classifiers, so there should be no need for you to implement one yourself.
Hmm, ... In my experience if you have some intuition you should follow it, because even it is was wrong it's a great opportunity for improving your understanding of how things work. I think I will hack together an implementation of Naive Bayes Mixture of Experts from scratch, just to see how it works, what features the experts use, etc.