I need some help in finding out precision matrix of a multivariate Gaussian distribution using Gibbs Sampling with a known mean. I tried generating samples using Wishart distribution as a prior distribution, but not able to get the correct output.

I derived the following expression for the scale matrix and the degrees of freedom that I used in the posterior distribution(Wishart distribution).

alt text

alt text

I am not sure whether these expression is correct or not.

Do I need to sample each component of the precision separately( by rearranging the multivariate expression into univariate expression), or the sampling from the wishart distribution will take care of it ?

asked Nov 26 '10 at 22:27

Saurabh%20Saxena's gravatar image

Saurabh Saxena

edited Nov 28 '10 at 18:19

A sample from the wishart is the conjugate prior to the inverse covariance, not to the covariance per se, I think.

(Nov 27 '10 at 04:05) Alexandre Passos ♦

One Answer:

For some reason what your derivation isn't showing up on my browser. Wikipedia has the parameters for the conjugate update: conjugate priors. Alexandre is right that the conjugate prior for the covariance matrix is the inverse Wishart (or the Wishart is the conjugate prior for the inverse covariance matrix). Essentially the updated covariance parameter to the inverse-Wishart distribution is the sum over the prior covariance matrix, outer product of each data point (centered at the sample mean), and the outer product of the sample mean (centered at the prior mean) weighted by the k0*N/(k0+N),

where k0 is the prior variance on the mean, and N is the number of data points.

The updated degrees of freedom of the inverse wishart is N + v0. (where v0 is the prior degrees of freedom). I think v0 has to be >= the dimensionality of the data, but i forget.

answered Nov 27 '10 at 14:14

Joseph%20Austerweil's gravatar image

Joseph Austerweil

Edited the question with the equation, there was some problem with the image hosting site.

(Nov 27 '10 at 14:32) Saurabh Saxena

ah yeah, I can see it now. there's an issue with your update that mu being sampled and needing to adjust your covariance update accordingly. Additionally, your sample from the wishart will be the inverse of the sample for your covariance matrix. have you had any luck fixing it?

(Nov 27 '10 at 16:12) Joseph Austerweil

There was a bug in my code, its working now

(Nov 28 '10 at 14:32) Saurabh Saxena
Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.