|
Hi, Being sorry if this question may seem easy and below the standards of the forum. That's because I'm new to the field. I have read the original LDA. In the original LDA it is mentioned that the coupling between theta and beta has produced the major computational difficulty. Later in the paper to do variational inference it relaxes the graphical model to solve the coupling. How shall I interpret coupling? Thank you. |
|
In LDA the fact that the z variables are not observed makes theta and beta correlated. If you write things down, you'll see that if you know the value of z then it is very easy to compute the posterior for theta or beta, as both are analytic integrals. Also, if you know the value of either theta or beta you can again analytically integrate out z and get a posterior distribution on the other, as again this case is very easy. Since you don't know either of them or z what happens is that any change in the distribution for theta will affect the posterior for z and beta, and likewise for all other possibilities. This is what makes inference in LDA hard. Thank you Alex for taking time to answer. That's kind of you. The last question I had asked here (http://metaoptimize.com/qa/questions/5533/is-it-possible-to-use-bayesian-networks-to-do-regression) that you responded forced me to go and watch all videos in free lectures by Lee Why teh, Volker Tresp, David Blei, Mike Jordan, Zoubin Ghahramani, Nando de Freitas, C.Bishop, and C.Robert that all had lots of useful data about Dirichlet, topic modeling, MCMC techniques, and a bit about variational Inference. After watching them and reading the useful stuff in this forum that lots of them were your posts and other friends even I prepared myself and gave a lecture based on some of the things that I had partially learned :) But I think I should read more about variational inference and MCMC techniques and their usefulness in different versions of LDA. It's really interesting. Could you please introduce me a list of other interesting papers that you yourself enjoined reading when you first encountered them and that they include LDA, Variational inference, or MCMC techniques. Thanks.
(Jul 06 '11 at 09:45)
Bahr
|
