|
I was jumping around generating dirichlets, and found out something odd, and I want to know if it is a problem with my definition or is it supposed to behave that way. Supposing that we have the basic PDF for a dirichlet distribution defined by: gamma(sum(ALPHA))/prod(gamma(ALPHA)))*prod(THETA.^(ALPHA-1)); And we have K=3 with parameters ALPHA =1 (The three of them), that gives us a distribution with the same probability over the simplex. My problem is that the value is 2 and I guessed it should be between 0 and 1. The ALPHA-1 factor goes to 0, making the product of the THETA's equal to 1. The sum over the alpha is 3, and the gamma of that equals 2. The gamma of each alpha equals, and the product equals 1. Thus leaving the result being equal to 2. Is this because we are working in the simplex and thus a different manifold with different metrics, or am I simpy doing it wrong? Thanks |
I don't exactly understand what you are doing. One thing that might explain it is that the dirichlet distribution, just like every other continuous distribution, is a density function, and can have any value larger than 0 at a point. The only constraint is that, when integrated over the simplex, it sums to one.
If you're not convinced this can happen, look at a normal distribution with mean 0 and variance 0.0001 and see the density value at the zero point.
But how can it integrate to one, if one of the values is higher than 1. Sorry if it is too basic, I just want to have a good grasp of it.
Integrals are not sums, you have to think about the area. Think of a rectangle 0.1 units wide and 10 units tall. Its area is 1. As long as the region of space that has density higher than 1 isn't too big it can still integrate to 1.
Ok, now I get it, my problem was that since I was thinking of dirichlet as being the prior to a discrete, I thought those values had to be less than one, but the dirichlet itself is continuous, it is the draws that describe a discrete.
Thanks
It's more complex. The draws of a dirichlet are probability distributions, and these can be the prior to a discrete variable. For an example, for that 3-variable dirichlet you described above, if the parameters are (0.1, 0.1, 0.1), a draw can be (0.003, 0.995, 0.002). These are probabilities, and if you draw a variable from these probabilities then you have something discrete.