|
Is there any rule of thumb (or empirical observation) for specifying parameters of Gibbs sampling for LDA? For example, "iteration"=2000, "burn-in"=0, "thinning"=2000 |
|
I don't use any burn-in or thinning interval and decide the number of iterations afteer seeing that the full-sample likelihood has converged (which depends on the data). If I'm sampling hyperparameters, seeing that they have converged is also good. I don't think it's always a good idea to go fully monte-carlo on LDA samplers, mainly due to unidentifiability, so I usually just take the last sample. I don't think I ever needed more than 50 iterations, and usually after two iterations a simple model is already reflecting the corpus quite a bit and can be interpreted. |
|
There are no defaults; it's highly dependent on the size of your corpus. I tend to let it run for as long as I can, and watch the likelihood (like Alexandre mentioned) to see if it stabilizes. Due to the unidentifiability issue of LDA, it's generally not a good idea to combine multiple samples, which makes thinning irrelevant. The last time I ran an LDA sampler, it was on a huge corpus (250k+ Twitter user timelines), and I let it go for about a week. The likelihood still hadn't stabilized by then, but the topics were very intelligible so I just stopped it. |
