|
Hi everyone, I'm a beginner to learn the LDA Algorithm. But I can't understand how to expand equation (14)
to equation (15)
in the original paper Latent Dirichlet Allocation(Blei et al., 2003). Just take the last expectation E_q[log q(z)] of (14) for example, in my understanding,
But how can I get the expectation like 5th line of equation (15)? Best regards, Richard |
|
Each of the terms in equation 14 expands to one line in equation 15. To compute these expectations you just write down the expansions of the formulas for the probability densities needed and use regular algebraic identities. For example, for the first case, E[log P(theta|alpha)], note that the density of theta is a Dirichlet, so it becomes
Expanding the logarithms you get
Using linearity of expectation then you get the first two terms in line 15. To get the last term, look up on wikipedia for the Dirichlet distribution the expectation of log theta_i, which is Digamma(alpha_i) - Digamma(sum_k alpha_k). Using the fact that the variational parameters for theta are named gamma then you have the final expression. For the other four lines in eq (15) you use essentially the same logic. Thank you very much! It is very helpful! May I ask one more question that what's the different between the expectation operator "E_q" in equation (14) and standard expectation operator "E"?
(Nov 01 '11 at 09:03)
Richard
E_q means an expectation over the variational distribution q, while E just means expectation and the distribution is implicit. In other words, E_q[f(x)] = int dq q(x) log f(x)
(Nov 01 '11 at 09:06)
Alexandre Passos ♦
Many thanks!
(Nov 01 '11 at 09:16)
Richard
|



which equation is 14 ? and which is 15 ? (there is not 5th line here... only 4..)
Sorry for missing the link of the original paper...Add it now...
What I want to know is how to expand the expectation from equation (14) to equation (15). Thanks a lot!