Gaussian scale mixture seems to be a good tool for modeling heavy-tailed residual in regression. I wonder if there is a systematic way to fit Gaussian scale mixture to data ? It appears that expectation-maximization algorithm only works for fitting location-scale mixture distribution, is that correct?

asked Aug 19 '11 at 14:52

Li%20Tang's gravatar image

Li Tang
1111


One Answer:

Here's an answer to a slightly different question. There exists ways of more directly defining a heavy tailed distribution other than using a mixture of gaussians (which are inherently not heavy-tailed). The challenge in the past has been efficient/compact representation, as well as learning/inference. There has been some work recently to address these issues, e.g., http://www.select.cs.cmu.edu/publications/paperdir/nips10-bickson-guestrin.pdf

This might better solve whatever problem it is you have.

answered Aug 20 '11 at 14:37

Yisong%20Yue's gravatar image

Yisong Yue
58631020

Thanks for your response. But the class of Gaussian scale mixture distributions contains many widely used heavy-tail distributions, e.g., the symmetric stable distributions and Laplace distributions. In fact, in the work you mentioned, "Inference with Multivariate Heavy-Tails in Linear Models", the authors assume that all observation nodes of their model are distributed according to a stable distribution. So intuitively, I think the class of Gaussian scale mixture would offer better fitting flexibility, although the learning/inference may be much harder.

(Aug 20 '11 at 15:20) Li Tang
Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.