Are there any tools for incremental training of svm models? It means that I need to continuously modify a model without complete retraining (or at least optimized retraining). I have found some articles that states a possibility of such an algorithm. Still I didn't find any implementations.


asked Feb 14 '12 at 04:40

Konstantin's gravatar image


3 Answers:

If the kernel is linear (i.e. no kernel) then Stochastic Gradient Descent will work (e.g. SGDClassifier in scikit-learn using the partial_fit method). If you want non linear kernels (e.g. RBF kernel) then you should have a look at LaSVM. I am not sure the commandline allows for incremental learning but as the algorithm is online it should be possible to do it if you use it as a library.

answered Feb 14 '12 at 04:55

ogrisel's gravatar image


edited Feb 14 '12 at 05:08

Thanks Olivier. I'm using linear kernel. I wanted to try scikit-learn before. Maybe that's a good time to start. I wonder if its performance (classification speed) comparable to libsvm/liblinear?

I'll take a look at LaSVM as well.

(Feb 14 '12 at 05:11) Konstantin

scikit-learn wraps libsvm in sklearn.svm.SVC and liblinear in sklearn.svm.LinearSVC but sklearn.linear_model.SGDClassifier should be faster at classification time as it does not copy the any data if the input is either a C-contiguous numpy array or a scipy.sparse.csr_matrix with dtype=numpy.float64.

(Feb 14 '12 at 05:16) ogrisel

Got it, thanks.

(Feb 14 '12 at 21:16) Konstantin

If you have very large scale data, vowpal wabbit can minimize the hinge loss with stochastic gradient descent.

answered Feb 14 '12 at 07:42

Alexandre%20Passos's gravatar image

Alexandre Passos ♦

Interesting, thanks. Though the project's wiki is not really verbose and it seems I didn't get that active learning concept. Could you point me to some papers describing it? I would be much obliged.

(Feb 14 '12 at 22:02) Konstantin

A good reference on active learning (that does not include the techniques in vowpal wabbit) is the following tutorial: http://hunch.net/~active_learning/

(Feb 14 '12 at 22:04) Alexandre Passos ♦

thanks, i'll take a look

(Feb 15 '12 at 02:34) Konstantin

There are online tools and some incremental tools as well, but to suit your needs you might need to hack trivially a bit. List of tools with remarks is presented here: http://stats.stackexchange.com/questions/30834/is-it-possible-to-append-training-data-to-existing-svm-models/51989#51989

answered Mar 13 '13 at 03:12

rahulkmishra's gravatar image


Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.