|
I'm thinking on making a toy implementation of this algorithm, following Bishop's book (p. 474) I am, however, a little bit confused in some variables: The parameters of the Gaussian Wishart Distribution: W_0 beta_0, how are those supposed to be initialized, is zero a valid value?? Which would be a good initial distribution r_nk to start the iterations, is any probabilisticaly valid array good enough. Thanks |
|
Ideally you should initialize the means with random vectors so you can do multiple restarts and choose the best lower bound. The hyperparameters of the inverse-wishart (if that is what you're talking about, I don't remember this part from Bishop very well) distribution shouldn't be zero. It is better to initialize them with something that probably overshoots the true value (so at first the means are reasonably free to move around) and then slowly optimize them (i.e., maximizing the lower bound according to their values, respecting the constraints) as the optimization progresses. |
|
I got a working python implementation based on Bishops chapter. Do you want that? I used random initializations for almost everything. But for W_0 I think I used the identity matrix, which I thought was a reasonable prior for the shape of a Gaussian. |
