Tools for SVM incremental training

1
1

Are there any tools for incremental training of svm models? It means that I need to continuously modify a model without complete retraining (or at least optimized retraining). I have found some articles that states a possibility of such an algorithm. Still I didn't find any implementations.

Thanks!

asked Feb 14 at 04:40

Konstantin's gravatar image

Konstantin
36●2●3●5
2 Answers:
oldest newest most voted
2

If the kernel is linear (i.e. no kernel) then Stochastic Gradient Descent will work (e.g. SGDClassifier in scikit-learn using the partial_fit method). If you want non linear kernels (e.g. RBF kernel) then you should have a look at LaSVM. I am not sure the commandline allows for incremental learning but as the algorithm is online it should be possible to do it if you use it as a library.

answered Feb 14 at 04:55

ogrisel's gravatar image

ogrisel
3984●6●44●80

edited Feb 14 at 05:08

Thanks Olivier. I'm using linear kernel. I wanted to try scikit-learn before. Maybe that's a good time to start. I wonder if its performance (classification speed) comparable to libsvm/liblinear?

I'll take a look at LaSVM as well.

(Feb 14 at 05:11) Konstantin

scikit-learn wraps libsvm in sklearn.svm.SVC and liblinear in sklearn.svm.LinearSVC but sklearn.linear_model.SGDClassifier should be faster at classification time as it does not copy the any data if the input is either a C-contiguous numpy array or a scipy.sparse.csr_matrix with dtype=numpy.float64.

(Feb 14 at 05:16) ogrisel

Got it, thanks.

(Feb 14 at 21:16) Konstantin
2

If you have very large scale data, vowpal wabbit can minimize the hinge loss with stochastic gradient descent.

answered Feb 14 at 07:42

Alexandre%20Passos's gravatar image

Alexandre Passos ♦
18997●44●214●335

Interesting, thanks. Though the project's wiki is not really verbose and it seems I didn't get that active learning concept. Could you point me to some papers describing it? I would be much obliged.

(Feb 14 at 22:02) Konstantin

A good reference on active learning (that does not include the techniques in vowpal wabbit) is the following tutorial: http://hunch.net/~active_learning/

(Feb 14 at 22:04) Alexandre Passos ♦

thanks, i'll take a look

(Feb 15 at 02:34) Konstantin
Your answer
toggle preview

learn more | privacy | contact

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.