why should we do gibbs sampling along with LDA, is that mandatory or the reson for it.I have seen some related posts in LDA that Arun Kumar,Leon Palafox, Alexandre passos, are guiding people in a nice way. I am new to topic modeling. So it will be nice if you people also teach me.

asked Sep 22 '13 at 05:36

ramy%20shanaran's gravatar image

ramy shanaran
16334


2 Answers:

Gibbs sampling is just one way of sampling from a joint probability distribution of two or more random variables. It is not mandatory for LDA.

David Blei has a paper that makes some changes to the original LDA method and uses variational Bayes. Online Learning for Latent Dirichlet Allocation

You can find it implemented on his website. Blei's Topic Modeling Website

answered Sep 23 '13 at 02:51

Eric%20Martinez's gravatar image

Eric Martinez
1112

Actually in the very first implementation of LDA, if I remember correctly, Blei used Variational inference to explain the topic.

Variational Inference tends to be faster than Gibbs Sampling, and tends to converge better than Gibbs Sampling. It is easy to do a coding mistake, and you will still get a reasonable output using Gibbs Sampling, because of the mixing, but with VB is harder to make those mistakes.

In a more profound theme, is easier to derive a Gibbs Sampler than the VB for most algorithms, sometimes VB require doing hard math and takes tons of time for things that are hardly a noticeable enhancement.

Alexandre summarized it better in this post: http://metaoptimize.com/qa/questions/836/the-choice-of-mcmc-and-variational-inference

Long story short, is up to you what to use, I just happen to like Gibbs Sampling, because I've used it for other things, and for most things I do VB is not the best option. But if you prefer VB, go ahead!

answered Oct 10 '13 at 16:13

Leon%20Palafox's gravatar image

Leon Palafox ♦
40857194128

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.