Is the log partition function only defined for exponential family distributions, or for any probability distribution in general?

asked Aug 03 '11 at 15:30

ebony's gravatar image

ebony
18181014


2 Answers:

It is defined for any possibly unnormalized (but normalizable) energy/density you want to build a probability distribution out of. Most usages of the term are actually, as you mentioned, related to Gibbs distributions (proportional to exp(- temperature * f(x))), which include exponential-family, but in things like mixture models (which are not in the exponential family) people still talk about log partition functions, as the logarithm of the integral of the probability of data given parameters times probability of the parameters over the parameters, so it's effectively the log likelihood of the data given the model.

answered Aug 04 '11 at 03:39

Alexandre%20Passos's gravatar image

Alexandre Passos ♦
2554154278421

The log partition function is not specific to exponential families. The partition function itself is more general, whereas the log partition function is talked about because of the form the exponential family distributions take. From the wikipedia article on exponential family distributions, they all take the form:

f(x|theta) = h(x)*exp( eta(theta)*T(x) - A(theta) )

For any distribution that is a product of multiple distinct components, you can attempt to put it into a form resembling the exponential families by taking the log of everything, and you'll end up with a quantity under the exponent that is the log partition function -- it just means A(theta) = log(Z(theta)), where Z is the the partition function.

answered Aug 03 '11 at 16:54

Brian%20Vandenberg's gravatar image

Brian Vandenberg
824213746

edited Aug 03 '11 at 16:56

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.