In your Opinion what are some of the best tutorials and survey papers on Gibbs Sampling out there.
I have read a lot, but more in the context of LDA and Dirichlet Process than as the sampling model itslef.
I know Bishop has a chapter where he discusses Gibbs Sampling, but I found it quite shallow in some aspects.
Do you know about a good tutorial in which they have other examples, or is it that Gibbs Sampling is only practical when using it with DP
asked Nov 16 '10 at 21:56
Leon Palafox ♦
Gibbs sampling is much more broadly applicable than just for DP or LDA. Basically, you can use it pretty much anywhere if you can write down the posterior of one of the variables conditioned on the rest (and do this in a cyclic fashion for all the variables).
The Gibbs sampling for the uninitiated tutorial by Philip Resnik and Eric Hardisty is reasonably detailed and quite good from a practical perspective (has lots of real examples worked out).
From the statistics point of view, some of the foundational papers (even the paper that defined it by Geman and Geman 1984 is pretty good) that show the generality of the Gibbs sampler are Gelfand 1990 and Casella 1992.
"Monte Carlo Statistical Methods" by Robert and Casella is a good book on sampling more generally and has a pretty thorough discussion of Gibbs sampling.
answered Nov 16 '10 at 22:39
Geoff Hinton's recorded lectures were also fairly enlightening. There's one on youtube, and 1 or 2 on videolectures.net. The latter is more or less the youtube video with a lot more detail, and about twice as long.
answered Dec 01 '10 at 14:28