|
Say 1 sample corresponds to 1 day of observations, and I make observations every hour on the fly(online), i.e. the snapshot taken at every hour is a feature(thus 1 day = 1 sample has 24 features). After every snapshot, I want to classify today's date as 'good' or 'bad', i.e. I am computing probabilistic distribution over these 2 labels. What I have now is 3 separate SVM models, one that does classification after 2 hours, one after 5 hours, and one after 10 hours of observations, and at each of these 3 checkpoints, uses SVM to compute the probability distribution over the 2 states. (some of these numbers are different in reality but you get the idea) Is there a better way to do this? Is there an equally simple classification method(framework) that outputs soft results(i.e. probabilities rather than just a label), but one that can easily give me results after every hour(i.e. after every single observation I make, rather than at specific checkpoints)? I would like to achieve this without making 24 separate models, obviously. |
|
While SVMs can be seen as giving you a probability, they are originally stated as an assignment over their position on a plane. With SVMs you are looking for the optimal plane that separates 2 sets (or more) of data. If you are looking for probabilities, I think it would be more intuitive to use something like Logistic Regression or Gaussian Processes, which are formulated on the terms of probabilities of data sets. Will they work online, with continuous updating of the output beliefs, as described in the question?
(Apr 23 '12 at 00:12)
Viktor Simjanoski
What I would use is Bayesian Logistic Regression (Gaussian Processes are already Bayessian). In Bayessian Models, you have a prior distribution, which you use to compute the probabilities of your observed data, you can do an online learning model by using the result of your training after 1 day as the prior for the training for 2 days. Check the chapter on Bayesian Logistic Regression on bishop's Pattern Recognition and Machine Learning, he has a pretty well designed example.
(Apr 23 '12 at 00:17)
Leon Palafox ♦
That sounds good, thanks a lot!
(Apr 23 '12 at 00:19)
Viktor Simjanoski
So, just to confirm, when I use Bayesian Logistic Regression, I will need to train on full samples(with 24 features), but I will be able to get classification probabilities on the fly, even with incomplete features?
(Apr 23 '12 at 10:04)
Viktor Simjanoski
|
