0
2

Hi,

I am using libSVM with Linear Kernel and liblinear to train/test on the same data. However, I receive different results. Why would that be ? I would ideally like to use liblinear as my feature matrix consists of 338 x 3866489 values.

asked May 16 '11 at 03:19

Dexter's gravatar image

Dexter
416243438

What do you mean when saying "different results" ? Do you get different weights ? How big is the relative difference in the weights ? Or do you get different classifiers concerning their results when applied to test data ?

(May 16 '11 at 12:34) Uwe Schmitt

Uwe, Sorry if I wasn't clear earlier. The accuracies are different. There's a difference of about 11% with libLinear being the higher one.

(May 16 '11 at 12:57) Dexter

2 Answers:

As http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf introduces the mathematica setting libsvm uses a L1-error term and quadratic (that is L2) regularization. Liblinear supports different combinations of L1/L2 errors and regularizations: Look at "introduction" at http://www.csie.ntu.edu.tw/~cjlin/liblinear.

So if you want to compare both approaches you should configure liblinear such that it fits libsvms setting. Despite of that difference you may get different results due to different algorithms. In a bad conditioned setting the weights might differ for nearly similiar values of the loss function.

answered May 16 '11 at 14:40

Uwe%20Schmitt's gravatar image

Uwe Schmitt
613

edited May 16 '11 at 17:17

Uwe, Thanks. I will look into this more.

(May 16 '11 at 14:54) Dexter

Here are some possibilities:

  1. different regularization constants (either literally or implicitly if they use slightly different forms)
  2. different convergence criteria
  3. different normalization
  4. different regularization terms (are both using L_2?)
  5. different loss functions (liblinear mentions L2-loss vs. L1-loss---I'm not 100% sure what they mean)
  6. different multi-class formulation (if your data is multi-class)

answered May 16 '11 at 14:04

jrennie's gravatar image

jrennie
12124

Jrennie, I think both have different objective functions. Let me dig into it more.

(May 16 '11 at 14:11) Dexter
Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.