|
Hi everyone, I am strugling with the imlpementation of a mixture of linear regressions. I follow the description in link text. Each expert is gaussian and the gates are softmax functions. I try to train the model with the weighted least square for the experts and least square for the gates... However I am unable to learn the gates probability distributions. I was hoping that someone would be able to point me to an implementation of MoE with the EM for a gaussian and softmax model (preferably in matlab or python). So far I was only able to find the Bayesian Toolbox for matlab on the net but it is huge and my goal is to compare the code and find my mistake. Thank you for your help. Edit: It turns out that the least square approximation proposed in link text works quite bad in the simple case I try to learn (I actually wonder if it works at all). I am now resorting to a Newton-Raphson method but again my implementation is instable and the hessian is often ill conditioned. Does anyone have a good reference on the implementation of the Newton-Raphson method. I am also looking for convergence condition in the case of the softmax regression -actually for minimizing a cross-entropy. Also I am still looking for an MoE implementation, if anyone knows one. Or multivariate logistic regression. |
