Tools for SVM incremental training

Are there any tools for incremental training of svm models? It means that I need to continuously modify a model without complete retraining (or at least optimized retraining). I have found some articles that states a possibility of such an algorithm. Still I didn't find any implementations.

Thanks!

asked Feb 14 '12 at 04:40

Konstantin
341●8●12●18

3 Answers:

oldest newest most voted

If the kernel is linear (i.e. no kernel) then Stochastic Gradient Descent will work (e.g. SGDClassifier in scikit-learn using the partial_fit method). If you want non linear kernels (e.g. RBF kernel) then you should have a look at LaSVM. I am not sure the commandline allows for incremental learning but as the algorithm is online it should be possible to do it if you use it as a library.

answered Feb 14 '12 at 04:55

ogrisel
4989●9●55●91

edited Feb 14 '12 at 05:08

Thanks Olivier. I'm using linear kernel. I wanted to try scikit-learn before. Maybe that's a good time to start. I wonder if its performance (classification speed) comparable to libsvm/liblinear?

I'll take a look at LaSVM as well.

(Feb 14 '12 at 05:11) Konstantin

scikit-learn wraps libsvm in sklearn.svm.SVC and liblinear in sklearn.svm.LinearSVC but sklearn.linear_model.SGDClassifier should be faster at classification time as it does not copy the any data if the input is either a C-contiguous numpy array or a scipy.sparse.csr_matrix with dtype=numpy.float64.

(Feb 14 '12 at 05:16) ogrisel

Got it, thanks.

(Feb 14 '12 at 21:16) Konstantin

If you have very large scale data, vowpal wabbit can minimize the hinge loss with stochastic gradient descent.

answered Feb 14 '12 at 07:42

Alexandre Passos ♦
25541●54●278●421

Interesting, thanks. Though the project's wiki is not really verbose and it seems I didn't get that active learning concept. Could you point me to some papers describing it? I would be much obliged.

(Feb 14 '12 at 22:02) Konstantin

A good reference on active learning (that does not include the techniques in vowpal wabbit) is the following tutorial: http://hunch.net/~active_learning/

(Feb 14 '12 at 22:04) Alexandre Passos ♦

thanks, i'll take a look

(Feb 15 '12 at 02:34) Konstantin

There are online tools and some incremental tools as well, but to suit your needs you might need to hack trivially a bit. List of tools with remarks is presented here: http://stats.stackexchange.com/questions/30834/is-it-possible-to-append-training-data-to-existing-svm-models/51989#51989

answered Mar 13 '13 at 03:12

rahulkmishra
16●1●1●3

Subscription:

Once you sign in you will be able to subscribe for any updates here

Tags:

svm ×134
training ×32
svm-software ×7

Asked: Feb 14 '12 at 04:40

Seen: 3,636 times

Last updated: Mar 13 '13 at 03:12

Tools for SVM incremental training

Subscription:

Related questions