|
I know of the auxiliary variable sampler for Gibbs sampling in non-conjugate models (basically, where the prior and likelihood distributions aren't conjugate to each other; e.g., Bayesian logistic regression). I have a model where the prior is Gaussian and the likelihood is Dirichlet. Has anyone used/seen the auxiliary variable sampler for Gibbs sampling in such a setting? If not, are there any other suggestions? |
|
As long as you can compute the likelihood function you can use slice sampling to resample the variables. AFAIK you can both do it in a Gibbs fashion and update each variable individually or try to use the multivariable slice sampler (although I've never implemented/tested it) to see how it goes. If you can't easily compute the posterior for that node in the graph you can also think up a good proposal distribution and use Metropolis-Hastings to update each variable individually. The only trick to getting both these approaches fast (as far as I know) is nevr computing the full likelihood, only the factors involving the variables you're actually resampling. (one last edit) Given the similarity to bayesian logistic regression you might want to try hamiltonian monte carlo (which uses gradient information directly to improve mixing of M-H steps). I've also never used it, but Mackay has code and examples in chap 30 of his book. Maybe the elliptical slice sampler might help you (given the gaussianity of your prior). |
|
I don't have experience of using auxiliary variable sampler in an non-conjugate setting. An alternative method, which you may have already been aware of, is the Jain-Neal split-merge for nonconjugate Dirichlet process mixture, you may find the paper as well as discussions here. Notice that the proposed method there requires conditional conjugate, which may or may not hold in your case. Alternative, you may even try to avoid Dirichlet distribution if it is possible in your case. Andrew Gelman had a blog discussing it. |
