|
I have a linear regression (say) model p(t|x;w) = N(t ; m , D); Being Bayesian, I can put a Gaussian prior on parameter w. However, I've realized for some models we can put Gaussian-Wishart hyperprior on the Gaussian to be 'more' Bayesian. Is this correct ? Are both of these two models valid Bayesian models ? It seems to me that we can always put hyperprior, hyperhyperprior,.......... because it will still be a valid probabilistic model. I am wondering what's the difference between putting a prior and putting the hyperprior on the prior. Are they both Bayesian ? |
|
There is actually nothing wrong to adding priors over priors. The things is just how much marginal performance increment are you going to get doing this. Presumably adding extra hyperpriors is also going to increase your computational time and unless you use conjugate functions, you are going to have more complex sampling functions. Especially for a linear regression, I do not think you have really to go that far. Thanks for the reply. My question is really They are both valid Bayesian models right? So theoretically I can add prior and hyperprior, hyperhyperprior.... and marginalize out all variables to get the marginal. It will still be a valid, well-defined Bayesian model ?
(Nov 22 '13 at 19:33)
Jing
Yes, it is still a Bayesian model. Adding to Leon's comment, when you see that your performance is sensitive to hyper-parameters, it is good to include hyper-priors into the model. If not, it will only be additional computational burden.
(Nov 23 '13 at 08:18)
Rakesh Chalasani
|
