|
Hi, I'm a BS student of CS interested in ML. Imagine that, we want to do a regression task. The training samples for instance, include pairs of the form ([X1,X2,X3,X4,X5], Y1). Where Xi's denote elements of sample feature vectors and Y's are outputs. Think of that, we know 4 out of 5 of the features are independent of each other and just one is dependent on some of the other features and we know that regression is an expansion to the classification. Also that, because features are not completely independent of each other, we can't use Naive Bayes. In the case of classification we know that Bayesian Networks are good choices to do this. Since they are still of lower complexity in comparison with the brute force way of applying Bayesian classifiers (i.e. where all the features are assumed to be dependent). What would we do in such circumstances where we have partial dependencies and we want to do regression? And we want to use Bayesian methodology to some extent. |
|
The answer that Alexandre Passos had posted before the crash of the database. "One solution is to use a discriminative Bayesian network for classification. For example, in this blog post I develop a variational algorithm for inference in a Bayesian network for regression in a multi-task setting (that is, there are many related regression tasks you want to learn). The key idea is that the probability distribution of the Y nodes should be something like Y_i ~ Normal(w^T X_i, sigma), where w is a parameter vector that can have any prior structure you want. Or you could use a nonparametric model such as a Gaussian process and put priors on the covariance function." Thanks for Alexandre's contribution. I'd also be grateful if somebody else have any more comments to add. Best Regards, Hamid. Thanks, I was about to repost that :-)
(Apr 21 '11 at 05:45)
Alexandre Passos ♦
|
