|
Hello, I'm in the final steps for the implementation of a Gibbs Sampler for DP, and I'm closely following this paper. (Really Good if you want to do a basic implementation!) The thing is, in the updating of the parameters it says that we need to update (or re-sample) the parameters for the means and var. My question is: This re-sampling is only dependent on the empirical mean and variance of the data, so does that means that as we get new data, we do the re-sample based on that empirical data? I looked and looked, and I found no place where the re-sample is dependent on the elements of a particular category (tables in the CRP) This means, that each re-sample is dependent only in the increasing richness of data, rather than the elements in the groups. Regards |
|
Have you read these slides, in particular slide 47 where they define the mean in terms of the global prior modified by the empirical mean of the cluster and similarly for the variance. From reading through the paper you linked too, it looks like they simply defined the initial distributions for mu_j and S_j. The Conjugate Prior page on Wikipedia also has a list of the priors that both the slides and Rasmussen's paper mention which depend on the empirical mean/variance of the data points assigned to a component rather than the full dataset. |
Have you had any success with this model? I dug through this more and used Rasmussen'1s older Infinite Gaussian Mixture Model as a reference and wrote a simple finite version that should address your main concerns regarding updating the parameters. I put it up here on github. Hopefully that helps clarify things.