Hey!

I am currently doing a short text classification task and I have two types of features:

1) Bag-of-Words Tfidf features

2) Topic Model LDA features

Each on it's own is performing pretty well (BOW better than LDA), but now the idea came up to combine both feature groups to improve the classification.

My BoW Tfidf features are L2 normalized and the LDA features sum to 1.

I have now simply combined the feature by concatenating them. So:

sample1 #tfidf1 #tfidf2 ..... #LDA1 ... #LDA500..

My Baseline for this approach is the best performing classification task for basic BoW, which in my case is a linear SVC.

With the combination I perform unfortunately a little worse than the Baseline. If I switch to multinomial Naive Bayes I can outperform the Baseline conducted as well with MNB, but this Baseline is worse than the SVC Baseline.

Maybe someone has some ideas, how I can do better. Maybe this simple combination approach is not the best anyways.

Thanks and many Regards, Philipp