What's the benefit of LDA? Why LDA so hot?

LDA works pretty good when you compare it to other document clustering methods.

It is a pretty decent generative process overall, and in his original paper, Blei clearly explained differences and advantages over different models.

It is far from perfect, and that is the reasion there are so many extension concerning its inference and the overall structure of the model.

It is a great way to model topics not only in documents, but it can also be applied to a whole different set of different problems.

LDA is not exactly a document clustering method, it's easier to think of it as a discrete matrix decomposition method, something like PCA or LSA but regularized and dealing directly with probabilities that works well for discrete data. Some big uses of LDA, I think, are:

1. browsing: if you have a huge document collection it can get a lot easier to navigate using topics than it would be using words. There are far less topics than words, and yet the topics retain a lot of the meaning of the documents, usually, and it ends up being very intuitive to alternate looking at the more representative documents of a topic then the more representative topics for a document.

2. dimensionality reduction: just like PCA and LSA, LDA can give you a lower-dimensional representation for the documents in your corpus, and this can be useful for many things. For example, if you're learning a classifier but have very little training data you can use LDA to learn features for your classifier, and if the classes happen close to the topic lines (things like subject, genre, etc, are usually better captured by topics than things like sentiment) you will probably see an improvement when using topic features as well as word features.

3. as a part of more complex models: This is, I think, where a lot of the attention to LDA comes from. LDA is a rather simple building block that makes sense to use in all sorts of discrete bayesian models. For example, summarization can be done better with the help of topics, bilingual word alignment, as a part of deeper analysis of history of ideas, exploring the impact of papers, the votes of congressmen, and many other things. Topic models are interesting for these applications because they can help you do a focused dimensionality reduction, that simplifies the rest of the model reducing the number of parameters you need to deal with.