online learning for text classification without initial vocabulary

Is there a way to train a text classifier with online learning when at the beginning there are not features, but these are added to the vocabulary (feature space) as new examples (with words that don't belong to the vocabulary) arrive?

asked Mar 07 '13 at 00:20

Eddie23r
6●6●6●9

One Answer:

oldest newest most voted

Yes:

In linear models (linear SVM, logistic regression, naive Bayes in log-space), zero coefficients do not affect the decision. So, when you see a batch of documents with new words, you can expand the vocabulary with the n "new" words in the batch, append n zeros to the coefficient matrix, and train. Similar tricks may apply to other models; in neural nets, append n random numbers to the input-to-hidden weights.
Use the hashing trick.

answered Mar 07 '13 at 05:59

larsmans
676●5●14●24

edited Mar 07 '13 at 10:21

Subscription:

Once you sign in you will be able to subscribe for any updates here

Tags:

classification ×237
text-classification ×66
online-learning ×34

Asked: Mar 07 '13 at 00:20

Seen: 868 times

Last updated: Mar 07 '13 at 10:21

online learning for text classification without initial vocabulary

Subscription:

Related questions