I'm wondering how many regression techniques are out there that can be used for non-linear regression. The first methods that come to my mind are:

  1. Gaussian process regression / Kernel Ridge Regression
  2. Support Vector regression
  3. K-nearest neighbor regression (like Nadaraya-Watson)

Any other methods? Preferably methods that employ kernels.

asked Aug 01 '13 at 11:26

Tom's gravatar image

Tom
71101214

There are many, but something obvious which isn't on your list is neural networks.

(Aug 01 '13 at 11:31) alto

Okay, excluding neural networks.

(Aug 01 '13 at 11:39) Tom

How about regression trees? Both random forests and boosting can produce nonlinear regressors, without necessarily using a kernel, though it is possible to boost kernel classifiers.

There's also the bayes point machine http://research.microsoft.com/apps/pubs/default.aspx?id=65611 , which is a bayesian version of support vector machines, and can be implemented as many averaged kernel perceptrons, and is easily adaptable to regression.

(Aug 01 '13 at 12:20) Alexandre Passos ♦

spline regression

(Aug 02 '13 at 07:01) digdug
Be the first one to answer this question!
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.