Regression confidence using SVMs in python

Question

Hi,

I'm using regression SVMs in python and I am wondering if there is any way to get a "confidence-measure" value for its predictions.

Previously, when using SVMs for binary classification, I was able to compute a confidence-type value from the 'margin'. Here is some pseudo-code showing how I got a confidence value:

# Begin pseudo-code
import svm as svmlib

prob = svmlib.svm_problem(labels, data)
param = svmlib.svm_parameter(svm_type=svmlib.C_SVC, \
    kernel_type = svmlib.RBF)
model = svmlib.svm_model(prob, param)

# get confidence
confidence = self.model.predict_values_raw(sample_to_classify)

I imagine that the further the new sample is from the training data, the worse the confidence, but I'm looking for a function that might help compute a reasonable estimate for this.

My (high-level) problem is as follows:

I have a function F(x), where x is a high-dimensional vector
F(x) can be computed but it is very slow
I want to train a regression SVM to approximate it
If I can find values of 'x' that have low prediction confidence, I can add these points and retrain (aka. active learning)

Has anyone obtained/used regression-SVM confidence/margin values before?

Accepted Answer

5

There is a technique for "hedging" predictions, which gives provably good measures of confidence levels for a range of different prediction methods. It's described in the paper Hedging Predictions in Machine Learning, by Gammerman and Vovk.

answered Mar 01 '11 at 14:39

Oscar Täckström
1459●10●27●43

Answer 2

I don't think an SVM is appropriate to this task for essentially two reasons: the first is that it, as you noticed, has no built-in notion of confidence for regression problems, and the second is that the kernel SVMs take time proportional to the number of training points to predict a test point, and this might be too slow.

To solve the first problem I'd recommend you use a gaussian process directly instead of a support vector machine, as gaussian processes can give you a confidence value for their predictions, and you can then threshold this appropriately. However, predicting in gaussian processes is even more costly than predicting with an SVM, so you should probably use some approximation. This page has software in many different languages for approximate inference in gaussian processes.

Regression confidence using SVMs in python

Subscription:

Related questions