|
This should be a basic component of Bayesian statistics. Assume a univariate random variable x is from the Normal distribution with unknown mean mu, and unknown precision lambda. That is x ~ Normal(x | mu, 1 / lambda) A conditional conjugate prior for it is Normal-Gamma distribution, i.e: mu, lambda ~ Normal(mu| m0, 1 / (k lambda)) Gamma(lambda | a, b) There are certainly other conditional conjugate priors for the unknown mean and variance, such as Normal-inverse-Chisquare and Normal-inverse-gamma. My question is, what is the marginal likelihood p(X), with X observations of x, i.e. p(X) = integral p(x|mu, lambda) p(mu, lambda) d mu,lambda Kevin Murphy has it summarized here. I followed most of the derivations and it looks all right. However, my experience with the equations of p(X) in that article is that, for some X, p(X) yield a positive number which is certainly wrong. Is there anyone having alternative derivation of the marginal likelihood? Or implemented code for it? Or any other suggestion on conjugate prior for Normal distribution and its usage, such as how to set up the hyperparameters? Any suggestion will be highly appreciated. Thanks. |
|
If I understand your problem correctly there's nothing wrong if the marginal likelihood is a number higher than 1. You are dealing with continuous distributions, so what you are computing is a probability density function, and the only constraints it has to obey are positivity and that its integral is one. Even the usual gaussian distribution is like this; the density of the value 0 with a 0-mean and very small (0.0001, for example) variance is a lot bigger than one (in this case around 3989). Is this what your question is? Hi Alexandre: You answered my question. I was testing a model using discrete variables and later moved onto continuous variables, but my mind was not in the same page and the pdf vs pmf thing just didn't get into my mind. Thanks.
(Oct 19 '10 at 11:57)
Denzel
|
